Oct 30 18:37:36.918698 ip-10-0-132-122 systemd[1]: Starting Kubernetes Kubelet... Oct 30 18:37:37.269538 ip-10-0-132-122 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:37:37.269538 ip-10-0-132-122 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 18:37:37.269538 ip-10-0-132-122 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:37:37.269538 ip-10-0-132-122 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:37:37.275230 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275153 2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:37.275230 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275229 2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:37.275230 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275233 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:37.275322 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275236 2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:37.275322 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275239 2578 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:37.275322 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275243 2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:37.275322 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275246 2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:37.275322 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275248 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:37.275322 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275252 2578 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:37.275322 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275255 2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:37.275322 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275257 2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:37.275322 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275260 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:37.275322 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275264 2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:37.275322 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275269 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:37.275322 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275272 2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:37.275322 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275275 2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:37.275322 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275278 2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:37.275322 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275280 2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:37.275322 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275283 2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:37.275322 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275286 2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:37.275322 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275288 2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:37.275322 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275297 2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275300 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275302 2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275305 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275307 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275310 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275313 2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275316 2578 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275319 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275321 2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275324 2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275327 2578 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275330 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275333 2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275335 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275338 2578 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275341 2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275343 2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275346 2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275348 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:37.275792 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275351 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:37.276247 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275353 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:37.276247 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275356 2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:37.276247 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275359 2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:37.276247 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275361 2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:37.276247 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275364 2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:37.276247 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275367 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:37.276247 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275369 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:37.276247 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275372 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:37.276247 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275374 2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:37.276247 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275379 2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:37.276247 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275383 2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:37.276247 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275386 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:37.276247 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275389 2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:37.276247 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275392 2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:37.276247 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275395 2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:37.276247 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275398 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:37.276247 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275400 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:37.276247 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275403 2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:37.276247 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275405 2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275408 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275411 2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275413 2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275416 2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275418 2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275421 2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275440 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275444 2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275448 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275452 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275456 2578 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275460 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275463 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275466 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275469 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275472 2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275474 2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275477 2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275481 2578 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:37.276723 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275483 2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275486 2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275488 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275491 2578 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275494 2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275497 2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275500 2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275502 2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275505 2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275508 2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275511 2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275514 2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275516 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275519 2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275522 2578 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275524 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275527 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275529 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275532 2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275534 2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:37.277202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275537 2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:37.277685 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275539 2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:37.277685 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275542 2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:37.277685 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275545 2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:37.277685 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275549 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:37.277685 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.275552 2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:37.278012 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.277999 2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:37.278012 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278012 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:37.278066 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278016 2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:37.278066 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278020 2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:37.278066 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278023 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:37.278066 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278025 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:37.278066 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278028 2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:37.278066 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278031 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:37.278066 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278034 2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:37.278066 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278037 2578 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:37.278066 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278039 2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:37.278066 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278042 2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:37.278066 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278044 2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:37.278066 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278047 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:37.278066 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278050 2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:37.278066 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278052 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:37.278066 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278055 2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:37.278066 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278058 2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:37.278066 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278060 2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:37.278066 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278063 2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:37.278066 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278066 2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:37.278569 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278068 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:37.278569 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278071 2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:37.278569 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278074 2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:37.278569 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278076 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:37.278569 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278079 2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:37.278569 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278081 2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:37.278569 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278084 2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:37.278569 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278088 2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:37.278569 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278092 2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:37.278569 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278095 2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:37.278569 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278097 2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:37.278569 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278100 2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:37.278569 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278103 2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:37.278569 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278105 2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:37.278569 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278108 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:37.278569 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278111 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:37.278569 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278113 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:37.278569 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278116 2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:37.278569 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278118 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278121 2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278123 2578 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278126 2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278128 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278131 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278133 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278136 2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278138 2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278140 2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278143 2578 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278146 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278148 2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278150 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278153 2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278157 2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278160 2578 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278162 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278165 2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278168 2578 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:37.279019 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278171 2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:37.279519 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278173 2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:37.279519 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278176 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:37.279519 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278179 2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:37.279519 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278182 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:37.279519 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278185 2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:37.279519 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278187 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:37.279519 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278190 2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:37.279519 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278193 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:37.279519 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278195 2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:37.279519 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278197 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:37.279519 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278200 2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:37.279519 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278203 2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:37.279519 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278205 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:37.279519 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278208 2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:37.279519 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278210 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:37.279519 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278213 2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:37.279519 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278215 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:37.279519 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278218 2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:37.279519 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278220 2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278223 2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278226 2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278228 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278231 2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278234 2578 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278236 2578 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278239 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278242 2578 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278245 2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278248 2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278251 2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278254 2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278257 2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278259 2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278262 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278264 2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278267 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278269 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278272 2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:37.279992 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278278 2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278282 2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278285 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278288 2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278291 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278294 2578 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278296 2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278299 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278818 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278826 2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278830 2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278834 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278837 2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278841 2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278845 2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278848 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278851 2578 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278855 2578 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278857 2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278861 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:37.280484 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278864 2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278867 2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278869 2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278872 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278875 2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278877 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278880 2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278882 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278885 2578 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278888 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278890 2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278893 2578 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278896 2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278898 2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278901 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278904 2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278906 2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278909 2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278912 2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278914 2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:37.280958 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278917 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:37.281471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278920 2578 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:37.281471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278922 2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:37.281471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278925 2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:37.281471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278927 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:37.281471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278930 2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:37.281471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278933 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:37.281471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278936 2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:37.281471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278938 2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:37.281471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278940 2578 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:37.281471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278943 2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:37.281471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278945 2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:37.281471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278948 2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:37.281471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278951 2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:37.281471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278955 2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:37.281471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278958 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:37.281471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278961 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:37.281471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278963 2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:37.281471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278966 2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:37.281471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278969 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278971 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278974 2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278976 2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278979 2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278981 2578 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278984 2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278987 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278990 2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278992 2578 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278995 2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.278998 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279000 2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279004 2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279007 2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279009 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279012 2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279014 2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279017 2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279020 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:37.281919 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279022 2578 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279025 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279028 2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279030 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279033 2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279036 2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279038 2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279041 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279043 2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279046 2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279049 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279051 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279054 2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279056 2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279059 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279061 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279064 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279067 2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279069 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279072 2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:37.282386 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279075 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279077 2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279080 2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279082 2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279085 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279088 2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279090 2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279093 2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279096 2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279098 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279102 2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279104 2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279107 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279109 2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.279111 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.279946 2578 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.279960 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.279967 2578 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.279972 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.279976 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.279980 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 18:37:37.282869 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.279984 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.279988 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.279992 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.279995 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.279999 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280003 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280006 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280009 2578 flags.go:64] FLAG: --cgroup-root="" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280012 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280015 2578 flags.go:64] FLAG: --client-ca-file="" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280018 2578 flags.go:64] FLAG: --cloud-provider="external" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280022 2578 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280027 2578 flags.go:64] FLAG: --cluster-domain="" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280030 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280033 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280036 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280040 2578 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280044 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280047 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280050 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280053 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280057 2578 flags.go:64] FLAG: --contention-profiling="false" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280060 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280063 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 18:37:37.283343 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280066 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280069 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280073 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280076 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280080 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280082 2578 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280086 2578 flags.go:64] FLAG: --enable-server="true" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280089 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280093 2578 flags.go:64] FLAG: --event-burst="100" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280096 2578 flags.go:64] FLAG: --event-qps="50" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280099 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280102 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280105 2578 flags.go:64] FLAG: --eviction-hard="" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280110 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280113 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280116 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280119 2578 flags.go:64] FLAG: --eviction-soft="" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280122 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280125 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280128 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280134 2578 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280137 2578 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280140 2578 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280143 2578 flags.go:64] FLAG: --feature-gates="" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280147 2578 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 18:37:37.284026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280150 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280154 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280157 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280160 2578 flags.go:64] FLAG: --healthz-port="10248" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280163 2578 flags.go:64] FLAG: --help="false" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280166 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280169 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280172 2578 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280175 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280179 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280182 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280185 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280188 2578 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280191 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280193 2578 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280196 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280199 2578 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280202 2578 flags.go:64] FLAG: --kube-reserved="" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280205 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280208 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280211 2578 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280213 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280216 2578 flags.go:64] FLAG: --lock-file="" Oct 30 18:37:37.284675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280220 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280223 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280230 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280236 2578 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280240 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280244 2578 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280247 2578 flags.go:64] FLAG: --logging-format="text" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280250 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280254 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280257 2578 flags.go:64] FLAG: --manifest-url="" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280259 2578 flags.go:64] FLAG: --manifest-url-header="" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280264 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280267 2578 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280271 2578 flags.go:64] FLAG: --max-pods="110" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280274 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280277 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280280 2578 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280283 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280286 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280289 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280292 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280299 2578 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280302 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280305 2578 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280308 2578 flags.go:64] FLAG: --pod-cidr="" Oct 30 18:37:37.285228 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280311 2578 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280315 2578 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280318 2578 flags.go:64] FLAG: --pods-per-core="0" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280321 2578 flags.go:64] FLAG: --port="10250" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280323 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280327 2578 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-039c428ae86592b74" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280330 2578 flags.go:64] FLAG: --qos-reserved="" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280333 2578 flags.go:64] FLAG: --read-only-port="10255" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280336 2578 flags.go:64] FLAG: --register-node="true" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280339 2578 flags.go:64] FLAG: --register-with-taints="" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280343 2578 flags.go:64] FLAG: --registry-burst="10" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280347 2578 flags.go:64] FLAG: --registry-qps="5" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280351 2578 flags.go:64] FLAG: --reserved-cpus="" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280355 2578 flags.go:64] FLAG: --reserved-memory="" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280359 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280362 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280365 2578 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280368 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280371 2578 flags.go:64] FLAG: --runonce="false" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280374 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280377 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280380 2578 flags.go:64] FLAG: --seccomp-default="false" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280383 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280386 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280389 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280392 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 18:37:37.285823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280395 2578 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280398 2578 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280401 2578 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280404 2578 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280407 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280410 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280413 2578 flags.go:64] FLAG: --system-cgroups="" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280416 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280422 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280435 2578 flags.go:64] FLAG: --tls-cert-file="" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280438 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280442 2578 flags.go:64] FLAG: --tls-min-version="" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280445 2578 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280448 2578 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280451 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280455 2578 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280458 2578 flags.go:64] FLAG: --v="2" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280463 2578 flags.go:64] FLAG: --version="false" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280469 2578 flags.go:64] FLAG: --vmodule="" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280473 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280476 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280612 2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280617 2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280620 2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:37.286472 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280623 2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280625 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280628 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280630 2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280633 2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280636 2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280639 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280642 2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280644 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280647 2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280650 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280653 2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280655 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280658 2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280661 2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280664 2578 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280667 2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280669 2578 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280672 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280674 2578 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:37.287084 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280677 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280679 2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280682 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280685 2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280687 2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280691 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280695 2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280698 2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280700 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280703 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280706 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280709 2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280711 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280713 2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280716 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280719 2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280721 2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280724 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280726 2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280729 2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:37.287572 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280732 2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280734 2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280737 2578 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280740 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280742 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280745 2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280747 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280750 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280752 2578 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280755 2578 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280757 2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280760 2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280762 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280765 2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280768 2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280770 2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280773 2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280775 2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280779 2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280782 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:37.288052 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280784 2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:37.288726 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280787 2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:37.288726 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280792 2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:37.288726 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280795 2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:37.288726 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280799 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:37.288726 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280802 2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:37.288726 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280805 2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:37.288726 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280807 2578 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:37.288726 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280810 2578 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:37.288726 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280813 2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:37.288726 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280815 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:37.288726 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280818 2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:37.288726 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280821 2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:37.288726 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280823 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:37.288726 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280826 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:37.288726 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280828 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:37.288726 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280831 2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:37.288726 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280834 2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:37.288726 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280836 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:37.288726 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280839 2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:37.289413 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280841 2578 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:37.289413 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280844 2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:37.289413 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280847 2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:37.289413 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280850 2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:37.289413 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280854 2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:37.289413 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280857 2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:37.289413 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280860 2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:37.289413 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280863 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:37.289413 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280866 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:37.289413 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280868 2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:37.289413 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280872 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:37.289413 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280874 2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:37.289413 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280877 2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:37.289413 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280880 2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:37.289413 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280883 2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:37.289413 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280886 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:37.289413 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280889 2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:37.289413 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280891 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:37.289413 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280894 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:37.290022 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280896 2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:37.290022 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280899 2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:37.290022 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280902 2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:37.290022 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280904 2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:37.290022 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.280930 2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:37:37.290022 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280965 2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:37.290022 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280969 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:37.290022 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280972 2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:37.290022 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280975 2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:37.290022 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280978 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:37.290022 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280980 2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:37.290022 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280983 2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:37.290022 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280985 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:37.290022 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280988 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:37.290022 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280990 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:37.290022 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280993 2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280996 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.280998 2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281001 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281003 2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281006 2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281009 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281011 2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281014 2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281017 2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281020 2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281022 2578 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281025 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281028 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281031 2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281034 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281036 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281039 2578 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281041 2578 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281044 2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:37.290416 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281046 2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:37.290942 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281049 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:37.290942 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281051 2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:37.290942 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281055 2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:37.290942 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281058 2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:37.290942 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281060 2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:37.290942 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281063 2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:37.290942 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281066 2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:37.290942 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281068 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:37.290942 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281072 2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:37.290942 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281074 2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:37.290942 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281078 2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:37.290942 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281082 2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:37.290942 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281085 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:37.290942 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281088 2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:37.290942 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281091 2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:37.290942 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281093 2578 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:37.290942 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281096 2578 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:37.290942 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281098 2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:37.290942 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281101 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:37.291397 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281104 2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:37.291397 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281106 2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:37.291397 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281109 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:37.291397 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281111 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:37.291397 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281113 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:37.291397 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281116 2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:37.291397 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281119 2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:37.291397 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281122 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:37.291397 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281124 2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:37.291397 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281127 2578 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:37.291397 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281129 2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:37.291397 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281132 2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:37.291397 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281135 2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:37.291397 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281139 2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:37.291397 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281142 2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:37.291397 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281147 2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:37.291397 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281150 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:37.291397 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281153 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:37.291397 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281156 2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:37.291871 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281158 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:37.291871 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281161 2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:37.291871 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281165 2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:37.291871 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281168 2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:37.291871 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281170 2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:37.291871 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281173 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:37.291871 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281175 2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:37.291871 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281178 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:37.291871 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281180 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:37.291871 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281183 2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:37.291871 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281185 2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:37.291871 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281188 2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:37.291871 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281190 2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:37.291871 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281193 2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:37.291871 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281196 2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:37.291871 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281198 2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:37.291871 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281201 2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:37.291871 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281204 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:37.291871 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281215 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281218 2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281220 2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281224 2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281227 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281229 2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281232 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281234 2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281237 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281240 2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281244 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281247 2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281249 2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281253 2578 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281255 2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281258 2578 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281260 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281263 2578 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.281266 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.289891 2578 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 18:37:37.292312 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.289908 2578 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.289920 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.289925 2578 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.289991 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.289996 2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.289999 2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290002 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290005 2578 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290008 2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290010 2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290013 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290016 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290019 2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290021 2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290024 2578 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290027 2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290030 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290032 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290035 2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290038 2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:37.292789 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290040 2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:37.293267 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290043 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:37.293267 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290045 2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:37.293267 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290048 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:37.293267 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290053 2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:37.293267 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290056 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:37.293267 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290060 2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:37.293267 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290064 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:37.293267 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290067 2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:37.293267 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290070 2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:37.293267 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290073 2578 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:37.293267 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290077 2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:37.293267 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290080 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:37.293267 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290082 2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:37.293267 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290085 2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:37.293267 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290088 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:37.293267 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290091 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:37.293267 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290093 2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:37.293267 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290096 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:37.293267 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290099 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:37.293721 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290102 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:37.293721 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290104 2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:37.293721 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290107 2578 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:37.293721 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290110 2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:37.293721 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290112 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:37.293721 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290115 2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:37.293721 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290117 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:37.293721 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290120 2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:37.293721 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290122 2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:37.293721 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290125 2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:37.293721 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290127 2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:37.293721 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290130 2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:37.293721 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290133 2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:37.293721 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290136 2578 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:37.293721 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290138 2578 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:37.293721 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290141 2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:37.293721 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290143 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:37.293721 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290146 2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:37.293721 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290149 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290151 2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290154 2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290157 2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290160 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290163 2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290166 2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290169 2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290172 2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290174 2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290177 2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290180 2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290182 2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290185 2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290188 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290190 2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290193 2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290195 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290198 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290200 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:37.294152 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290203 2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:37.294821 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290205 2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:37.294821 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290207 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:37.294821 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290210 2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:37.294821 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290214 2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:37.294821 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290217 2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:37.294821 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290219 2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:37.294821 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290222 2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:37.294821 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290224 2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:37.294821 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290227 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:37.294821 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290229 2578 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:37.294821 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290232 2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:37.294821 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290235 2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:37.294821 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290237 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:37.294821 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290240 2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:37.294821 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290243 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:37.294821 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290246 2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:37.294821 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290248 2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:37.294821 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290252 2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:37.294821 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290255 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:37.295265 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290257 2578 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:37.295265 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290260 2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:37.295265 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290263 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:37.295265 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290266 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:37.295265 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290268 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:37.295265 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290270 2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:37.295265 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290275 2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:37.295265 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290278 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:37.295265 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290281 2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:37.295265 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290284 2578 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:37.295265 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290287 2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:37.295265 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.290316 2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:37:37.295265 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290490 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:37.295265 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290496 2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:37.295265 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290499 2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:37.295265 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290502 2578 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:37.295659 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290506 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:37.295659 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290509 2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:37.295659 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290512 2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:37.295659 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290515 2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:37.295659 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290518 2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:37.295659 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290520 2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:37.295659 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290523 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:37.295659 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290526 2578 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:37.295659 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290528 2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:37.295659 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290531 2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:37.295659 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290534 2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:37.295659 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290537 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:37.295659 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290539 2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:37.295659 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290542 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:37.295659 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290545 2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:37.295659 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290548 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:37.295659 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290551 2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:37.295659 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290553 2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:37.295659 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290556 2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:37.296106 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290560 2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:37.296106 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290564 2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:37.296106 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290567 2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:37.296106 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290570 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:37.296106 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290573 2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:37.296106 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290575 2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:37.296106 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290578 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:37.296106 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290581 2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:37.296106 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290583 2578 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:37.296106 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290586 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:37.296106 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290589 2578 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:37.296106 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290591 2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:37.296106 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290594 2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:37.296106 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290596 2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:37.296106 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290599 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:37.296106 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290602 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:37.296106 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290605 2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:37.296106 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290607 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290610 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290612 2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290615 2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290617 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290620 2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290622 2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290625 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290628 2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290630 2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290633 2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290636 2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290639 2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290643 2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290646 2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290648 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290651 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290653 2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290656 2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290658 2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:37.296580 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290661 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290664 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290667 2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290669 2578 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290676 2578 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290679 2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290682 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290684 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290687 2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290690 2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290692 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290695 2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290697 2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290700 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290702 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290705 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290707 2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290710 2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290713 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290715 2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:37.297045 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290718 2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290721 2578 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290723 2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290726 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290728 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290731 2578 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290733 2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290736 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290739 2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290741 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290743 2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290746 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290748 2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290751 2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290754 2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290756 2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290759 2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290762 2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290764 2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290767 2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:37.297575 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290769 2578 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:37.298029 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290772 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:37.298029 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290774 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:37.298029 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290777 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:37.298029 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290780 2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:37.298029 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290782 2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:37.298029 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.290808 2578 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:37:37.298029 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290840 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:37:37.298029 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290844 2578 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:37:37.298029 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290847 2578 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:37:37.298029 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290850 2578 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:37:37.298029 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290852 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:37:37.298029 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290855 2578 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:37:37.298029 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290858 2578 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:37:37.298029 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290860 2578 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:37:37.298029 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290863 2578 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:37:37.298029 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290865 2578 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:37:37.298405 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290868 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:37:37.298405 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290870 2578 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:37:37.298405 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290891 2578 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:37:37.298405 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290895 2578 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:37:37.298405 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290898 2578 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:37:37.298405 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290901 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:37:37.298405 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290904 2578 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:37:37.298405 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290907 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:37:37.298405 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290909 2578 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:37:37.298405 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290912 2578 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:37:37.298405 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290914 2578 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:37:37.298405 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290917 2578 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:37:37.298405 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290919 2578 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:37:37.298405 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290923 2578 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:37:37.298405 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290927 2578 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:37:37.298405 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290930 2578 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:37:37.298405 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290933 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:37:37.298405 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290936 2578 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:37:37.298405 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290939 2578 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:37:37.299003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290942 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:37:37.299003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290945 2578 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:37:37.299003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290948 2578 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:37:37.299003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290952 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:37:37.299003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290954 2578 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:37:37.299003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290957 2578 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:37:37.299003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290960 2578 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:37:37.299003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290962 2578 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:37:37.299003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290965 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:37:37.299003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290968 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:37:37.299003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290970 2578 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:37:37.299003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290973 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:37:37.299003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290976 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:37:37.299003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290978 2578 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:37:37.299003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290981 2578 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:37:37.299003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290983 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:37:37.299003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290986 2578 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:37:37.299003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290988 2578 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:37:37.299003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290991 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290994 2578 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290996 2578 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.290999 2578 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291001 2578 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291005 2578 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291009 2578 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291011 2578 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291014 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291017 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291019 2578 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291022 2578 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291025 2578 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291027 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291030 2578 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291033 2578 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291036 2578 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291039 2578 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291042 2578 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291045 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:37:37.299499 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291048 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291050 2578 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291053 2578 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291055 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291058 2578 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291060 2578 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291063 2578 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291065 2578 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291068 2578 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291070 2578 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291073 2578 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291078 2578 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291081 2578 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291083 2578 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291086 2578 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291089 2578 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291091 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291094 2578 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291096 2578 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291099 2578 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:37:37.299967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291101 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:37:37.300471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291105 2578 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:37:37.300471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291107 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:37:37.300471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291110 2578 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:37:37.300471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291112 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:37:37.300471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291115 2578 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:37:37.300471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291117 2578 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:37:37.300471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291119 2578 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:37:37.300471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291122 2578 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:37:37.300471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291125 2578 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:37:37.300471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291128 2578 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:37:37.300471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291130 2578 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:37:37.300471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291133 2578 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:37:37.300471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291135 2578 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:37:37.300471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291138 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:37:37.300471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291140 2578 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:37:37.300471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291143 2578 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:37:37.300471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291145 2578 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:37:37.300471 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:37.291148 2578 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:37:37.300471 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.291205 2578 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 18:37:37.300910 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.296113 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Oct 30 18:37:37.300910 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.297163 2578 server.go:1014] "Starting client certificate rotation" Oct 30 18:37:37.300910 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.297257 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 18:37:37.300910 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.297298 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 18:37:37.300910 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.299291 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:37:37.304193 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.304174 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:37:37.315140 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.315124 2578 log.go:25] "Validated CRI v1 runtime API" Oct 30 18:37:37.319710 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.319689 2578 log.go:25] "Validated CRI v1 image API" Oct 30 18:37:37.320534 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.320510 2578 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 18:37:37.324414 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.324393 2578 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b9b99cef-be28-4f4c-997a-4810993ee868:/dev/nvme0n1p4 c588d081-9b52-4f1d-802f-6799137d242e:/dev/nvme0n1p3] Oct 30 18:37:37.324490 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.324412 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Oct 30 18:37:37.330597 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.330481 2578 manager.go:217] Machine: {Timestamp:2026-10-30 18:37:37.329234699 +0000 UTC m=+0.323983037 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3090223 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2270c7bfbc8ca9ba20f574def71eb6 SystemUUID:ec2270c7-bfbc-8ca9-ba20-f574def71eb6 BootID:46453f11-dfb1-4507-a8f2-56c841df8ab8 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:66:a7:28:44:0f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:66:a7:28:44:0f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ae:64:8a:d1:2d:7e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 18:37:37.330687 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.330599 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 18:37:37.330687 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.330681 2578 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 18:37:37.331993 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.331973 2578 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 18:37:37.332146 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.331994 2578 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-122.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 18:37:37.332194 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.332152 2578 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 18:37:37.332194 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.332160 2578 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 18:37:37.332194 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.332173 2578 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:37:37.333202 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.333192 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:37:37.334453 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.334443 2578 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 18:37:37.339405 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.339390 2578 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 18:37:37.339539 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.339529 2578 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 18:37:37.340363 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.340355 2578 kubelet.go:493] "Attempting to sync node with API server" Oct 30 18:37:37.340396 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.340371 2578 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 18:37:37.340396 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.340381 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 18:37:37.340396 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.340390 2578 kubelet.go:405] "Adding apiserver pod source" Oct 30 18:37:37.340505 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.340397 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 18:37:37.340731 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.340719 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:37:37.343007 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.342989 2578 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 18:37:37.349964 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.349952 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:37:37.351005 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.350993 2578 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 18:37:37.351041 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.351024 2578 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 18:37:37.351176 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.351167 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 18:37:37.351208 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.351183 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 18:37:37.351208 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.351190 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 18:37:37.351208 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.351197 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 18:37:37.351208 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.351203 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 18:37:37.351307 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.351211 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 18:37:37.352061 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.352052 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 18:37:37.352093 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.352061 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 18:37:37.352093 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.352069 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 18:37:37.352093 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.352074 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 18:37:37.352093 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.352083 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 18:37:37.352093 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.352091 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 18:37:37.352874 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.352864 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 18:37:37.352910 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.352878 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 18:37:37.356676 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.356664 2578 server.go:1263] "Started kubelet" Oct 30 18:37:37.356814 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.356766 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 18:37:37.356814 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.356793 2578 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 18:37:37.356979 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.356853 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 18:37:37.357451 ip-10-0-132-122 systemd[1]: Started Kubernetes Kubelet. Oct 30 18:37:37.359662 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.359643 2578 server.go:317] "Adding debug handlers to kubelet server" Oct 30 18:37:37.360616 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.360588 2578 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 18:37:37.363393 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:37.363371 2578 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Oct 30 18:37:37.367510 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.367497 2578 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 18:37:37.367601 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.367584 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 18:37:37.367797 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:37.367773 2578 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-122.us-west-1.compute.internal\" not found" Oct 30 18:37:37.368221 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.368207 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 18:37:37.368336 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.368219 2578 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 18:37:37.368440 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.368410 2578 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 18:37:37.368524 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.368510 2578 reconstruct.go:97] "Volume reconstruction finished" Oct 30 18:37:37.368524 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.368519 2578 reconciler.go:29] "Reconciler: start to sync state" Oct 30 18:37:37.368802 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.368786 2578 factory.go:153] Registering CRI-O factory Oct 30 18:37:37.368802 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.368804 2578 factory.go:223] Registration of the crio container factory successfully Oct 30 18:37:37.368896 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.368844 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 18:37:37.368896 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.368853 2578 factory.go:55] Registering systemd factory Oct 30 18:37:37.368896 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.368859 2578 factory.go:223] Registration of the systemd container factory successfully Oct 30 18:37:37.368896 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.368874 2578 factory.go:103] Registering Raw factory Oct 30 18:37:37.368896 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.368891 2578 manager.go:1199] Started watching for new ooms in manager Oct 30 18:37:37.369985 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.369971 2578 manager.go:319] Starting recovery of all containers Oct 30 18:37:37.381291 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.381275 2578 manager.go:324] Recovery completed Oct 30 18:37:37.385264 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.385249 2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:37:37.387396 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.387369 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:37:37.387492 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.387412 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:37:37.387492 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.387441 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientPID" Oct 30 18:37:37.388047 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.388027 2578 cpu_manager.go:226] "Starting" policy="none" Oct 30 18:37:37.388047 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.388040 2578 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 18:37:37.388047 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.388046 2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:37:37.388209 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.388059 2578 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 18:37:37.388772 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.388758 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:37:37.388815 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.388784 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:37:37.388815 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.388793 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientPID" Oct 30 18:37:37.390302 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.390290 2578 policy_none.go:50] "Start" Oct 30 18:37:37.390346 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.390308 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 18:37:37.390346 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.390323 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 18:37:37.392445 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.392420 2578 policy_none.go:44] "Start" Oct 30 18:37:37.416375 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.416330 2578 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 18:37:37.443839 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.418723 2578 manager.go:341] "Starting Device Plugin manager" Oct 30 18:37:37.443839 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:37.418756 2578 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 18:37:37.443839 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.418765 2578 server.go:87] "Starting device plugin registration server" Oct 30 18:37:37.443839 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.419017 2578 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 18:37:37.443839 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.419030 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 18:37:37.443839 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.419196 2578 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 18:37:37.443839 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.419274 2578 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 18:37:37.443839 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.419282 2578 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 18:37:37.443839 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:37.419890 2578 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 18:37:37.443839 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:37.419937 2578 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-122.us-west-1.compute.internal\" not found" Oct 30 18:37:37.443839 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.432077 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 18:37:37.443839 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.433296 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 18:37:37.443839 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.433316 2578 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 18:37:37.443839 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.433336 2578 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 18:37:37.443839 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:37.433373 2578 kubelet.go:2545] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Oct 30 18:37:37.502247 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:37.502219 2578 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-122.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Oct 30 18:37:37.503293 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:37.502211 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-122.us-west-1.compute.internal.18e362925b17e94e default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-122.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-122.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 18:37:37.35663035 +0000 UTC m=+0.351378688,LastTimestamp:2026-10-30 18:37:37.35663035 +0000 UTC m=+0.351378688,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-122.us-west-1.compute.internal,}" Oct 30 18:37:37.519846 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.519808 2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:37:37.521504 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.521488 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:37:37.521583 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.521515 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:37:37.521583 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.521532 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientPID" Oct 30 18:37:37.521583 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.521559 2578 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.534501 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.534480 2578 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal"] Oct 30 18:37:37.534578 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.534535 2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:37:37.535256 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.535234 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:37:37.535331 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.535266 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:37:37.535331 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.535280 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientPID" Oct 30 18:37:37.537599 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.537471 2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:37:37.537652 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.537603 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.537652 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.537637 2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:37:37.538351 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.538334 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:37:37.538459 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.538355 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:37:37.538459 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.538365 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:37:37.538459 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.538379 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientPID" Oct 30 18:37:37.538459 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.538380 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:37:37.538459 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.538396 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientPID" Oct 30 18:37:37.540501 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.540487 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.540577 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.540511 2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:37:37.541531 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.541514 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:37:37.541622 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.541544 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:37:37.541622 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.541560 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientPID" Oct 30 18:37:37.551356 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.551342 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-122.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Oct 30 18:37:37.573947 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:37.573930 2578 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-122.us-west-1.compute.internal\" not found" node="ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.574719 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.574703 2578 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 18:37:37.574771 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.574742 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5vj22" Oct 30 18:37:37.576347 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:37.576330 2578 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-122.us-west-1.compute.internal\" not found" node="ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.590466 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:37.590365 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-122.us-west-1.compute.internal.18e362925ced5e5d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-122.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-122.us-west-1.compute.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-122.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 18:37:37.387396701 +0000 UTC m=+0.382145037,LastTimestamp:2026-10-30 18:37:37.387396701 +0000 UTC m=+0.382145037,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-122.us-west-1.compute.internal,}" Oct 30 18:37:37.590598 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:37.590483 2578 reflector.go:204] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" Oct 30 18:37:37.590598 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:37.590540 2578 reflector.go:204] "Failed to watch" err="failed to list *v1.RuntimeClass: runtimeclasses.node.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"runtimeclasses\" in API group \"node.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" Oct 30 18:37:37.590714 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:37.590646 2578 reflector.go:204] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" Oct 30 18:37:37.662273 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:37.662251 2578 kubelet_node_status.go:115] "Unable to register node with API server, error getting existing node" err="nodes \"ip-10-0-132-122.us-west-1.compute.internal\" not found" node="ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.663256 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.663238 2578 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:37:37.669644 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.669627 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7dbda9e04d721f174c9b45daae509010-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal\" (UID: \"7dbda9e04d721f174c9b45daae509010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.669713 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.669653 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7dbda9e04d721f174c9b45daae509010-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal\" (UID: \"7dbda9e04d721f174c9b45daae509010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.669713 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.669688 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/99a39564afa8d640f032873b7cd4dd97-config\") pod \"kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal\" (UID: \"99a39564afa8d640f032873b7cd4dd97\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.770238 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.770186 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7dbda9e04d721f174c9b45daae509010-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal\" (UID: \"7dbda9e04d721f174c9b45daae509010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.770349 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.770244 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7dbda9e04d721f174c9b45daae509010-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal\" (UID: \"7dbda9e04d721f174c9b45daae509010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.770349 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.770210 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7dbda9e04d721f174c9b45daae509010-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal\" (UID: \"7dbda9e04d721f174c9b45daae509010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.770349 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.770317 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/99a39564afa8d640f032873b7cd4dd97-config\") pod \"kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal\" (UID: \"99a39564afa8d640f032873b7cd4dd97\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.770349 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.770269 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7dbda9e04d721f174c9b45daae509010-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal\" (UID: \"7dbda9e04d721f174c9b45daae509010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.770512 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.770352 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/99a39564afa8d640f032873b7cd4dd97-config\") pod \"kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal\" (UID: \"99a39564afa8d640f032873b7cd4dd97\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.807040 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:37.807019 2578 nodelease.go:50] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-132-122.us-west-1.compute.internal\" not found" node="ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.862638 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.862618 2578 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:37:37.863516 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.863500 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:37:37.863602 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.863533 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:37:37.863602 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.863550 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientPID" Oct 30 18:37:37.863602 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.863589 2578 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.879922 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.879902 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.882067 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.882050 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.921447 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.921416 2578 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:37.967837 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:37.967816 2578 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:38.036319 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:38.036267 2578 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:37:38.037407 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:38.037394 2578 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:37:38.134235 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:38.134215 2578 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:37:39.578416 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.578328 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Oct 30 18:37:39.578909 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:39.578479 2578 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://aec3e5e704cd849e78ca3402ce7e1bb4-9d227f372467f59d.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-132-122.us-west-1.compute.internal.18e362925cedb898\": read tcp 10.0.132.122:38760->3.139.27.93:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-132-122.us-west-1.compute.internal.18e362925cedb898 default 4608 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-122.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-122.us-west-1.compute.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-122.us-west-1.compute.internal,},FirstTimestamp:2026-10-30 18:37:37 +0000 UTC,LastTimestamp:2026-10-30 18:37:37.535272826 +0000 UTC m=+0.530021167,Count:4,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-122.us-west-1.compute.internal,}" Oct 30 18:37:39.622505 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.622485 2578 apiserver.go:52] "Watching apiserver" Oct 30 18:37:39.649726 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.649704 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Oct 30 18:37:39.862960 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.862875 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:32:38 +0000 UTC" deadline="2028-04-25 06:13:40.809673716 +0000 UTC" Oct 30 18:37:39.862960 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.862910 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13019h36m0.946766778s" Oct 30 18:37:39.887871 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.887843 2578 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 18:37:39.888153 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.888135 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-95g7n","kube-system/global-pull-secret-syncer-hmzzs","openshift-multus/multus-m5xnp","openshift-network-operator/iptables-alerter-lnjwb","openshift-ovn-kubernetes/ovnkube-node-ls845","kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal","kube-system/konnectivity-agent-bchfv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v","openshift-cluster-node-tuning-operator/tuned-mc5j7","openshift-multus/multus-additional-cni-plugins-bh8c5","openshift-dns/node-resolver-5mmnd","openshift-image-registry/node-ca-kjn4r","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal","openshift-multus/network-metrics-daemon-2s5rm"] Oct 30 18:37:39.890656 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.890640 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:37:39.890738 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:39.890716 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95g7n" podUID="cd195fdf-8022-4279-8eca-15d145d59038" Oct 30 18:37:39.892683 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.892670 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:39.892739 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:39.892722 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hmzzs" podUID="8c9b04e2-b099-4704-8786-b64a8991d0f4" Oct 30 18:37:39.894667 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.894654 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m5xnp" Oct 30 18:37:39.896717 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.896702 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lnjwb" Oct 30 18:37:39.898866 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.898839 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.900904 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.900887 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bchfv" Oct 30 18:37:39.903022 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.903003 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:39.905181 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.905166 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:39.907331 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.907316 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:39.909524 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.909506 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5mmnd" Oct 30 18:37:39.912646 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.912633 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kjn4r" Oct 30 18:37:39.914807 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.914792 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:39.914886 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:39.914840 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2s5rm" podUID="8d3a00e6-202e-4ffb-b8e0-67f15d831ae2" Oct 30 18:37:39.945394 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.945372 2578 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:37:39.946679 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.946662 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 18:37:39.946679 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.946665 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 18:37:39.946792 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.946678 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.946792 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.946739 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.946928 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.946912 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-nsz9w\"" Oct 30 18:37:39.948318 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.948264 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.948318 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.948307 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.948477 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.948296 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 18:37:39.948477 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.948335 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-s4fk8\"" Oct 30 18:37:39.950544 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.950527 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 18:37:39.950632 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.950599 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.950632 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.950618 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 18:37:39.950698 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.950650 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.950698 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.950691 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-476j6\"" Oct 30 18:37:39.950752 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.950682 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 18:37:39.950752 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.950739 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 18:37:39.950806 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.950797 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 18:37:39.951941 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.951929 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 18:37:39.951986 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.951949 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-xlrq8\"" Oct 30 18:37:39.955179 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955164 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/15f5b642-cd06-42ad-9832-dcfe438ea4e5-serviceca\") pod \"node-ca-kjn4r\" (UID: \"15f5b642-cd06-42ad-9832-dcfe438ea4e5\") " pod="openshift-image-registry/node-ca-kjn4r" Oct 30 18:37:39.955224 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955186 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l7k8\" (UniqueName: \"kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8\") pod \"network-check-target-95g7n\" (UID: \"cd195fdf-8022-4279-8eca-15d145d59038\") " pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:37:39.955224 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955203 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8c9b04e2-b099-4704-8786-b64a8991d0f4-kubelet-config\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:39.955224 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955219 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-tuned\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:39.955311 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955235 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thmjv\" (UniqueName: \"kubernetes.io/projected/6b091344-3cff-4f72-9512-02499eb9a9ef-kube-api-access-thmjv\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:39.955344 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955320 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-etc-kubernetes\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:39.955375 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955342 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.955375 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955359 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-run\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:39.955437 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955374 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2bfeb3c6-d71a-4b5e-96db-198da488f7a9-hosts-file\") pod \"node-resolver-5mmnd\" (UID: \"2bfeb3c6-d71a-4b5e-96db-198da488f7a9\") " pod="openshift-dns/node-resolver-5mmnd" Oct 30 18:37:39.955437 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955388 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15f5b642-cd06-42ad-9832-dcfe438ea4e5-host\") pod \"node-ca-kjn4r\" (UID: \"15f5b642-cd06-42ad-9832-dcfe438ea4e5\") " pod="openshift-image-registry/node-ca-kjn4r" Oct 30 18:37:39.955437 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955424 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-kubelet\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.955531 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955456 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-registration-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:39.955531 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955478 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-lib-modules\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:39.955531 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955505 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-system-cni-dir\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:39.955531 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955526 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:39.955662 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955541 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-run-openvswitch\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.955662 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955586 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-sysctl-d\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:39.955662 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955607 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:39.955662 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955624 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-run-k8s-cni-cncf-io\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:39.955662 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955638 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-conf-dir\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:39.955793 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955660 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-node-log\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.955793 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955691 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-ovnkube-script-lib\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.955793 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955707 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-socket-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:39.955793 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955734 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b091344-3cff-4f72-9512-02499eb9a9ef-tmp\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:39.955793 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955761 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-systemd-units\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.955793 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955776 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-cni-binary-copy\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:39.955954 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955802 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-sysctl-conf\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:39.955954 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955818 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-systemd\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:39.955954 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955833 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ztsmk\" (UniqueName: \"kubernetes.io/projected/e2455903-bb15-41e2-8d91-51df8775b033-kube-api-access-ztsmk\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:39.955954 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955848 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-modprobe-d\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:39.955954 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955883 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-cnibin\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:39.955954 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955903 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-socket-dir-parent\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:39.955954 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955921 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-var-lib-cni-bin\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:39.955954 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955936 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-var-lib-cni-multus\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:39.956176 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955958 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-run-multus-certs\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:39.956176 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955972 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fkqwc\" (UniqueName: \"kubernetes.io/projected/c89d1725-2dbb-49dc-92b1-218935ce8a74-kube-api-access-fkqwc\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:39.956176 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.955998 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-run-ovn\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.956176 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956025 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-run-netns\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:39.956176 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956050 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-run-ovn-kubernetes\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.956176 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956072 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-cni-netd\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.956176 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956096 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs\") pod \"network-metrics-daemon-2s5rm\" (UID: \"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2\") " pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:39.956176 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956123 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7eed8239-b695-41f7-968f-b3a02659b44c-host-slash\") pod \"iptables-alerter-lnjwb\" (UID: \"7eed8239-b695-41f7-968f-b3a02659b44c\") " pod="openshift-network-operator/iptables-alerter-lnjwb" Oct 30 18:37:39.956176 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956147 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:39.956176 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956166 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-device-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:39.956435 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956180 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-system-cni-dir\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:39.956435 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956203 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-run-systemd\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.956435 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956226 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-log-socket\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.956435 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956249 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-sysconfig\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:39.956435 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956270 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-sys\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:39.956435 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956298 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-var-lib-kubelet\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:39.956435 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956323 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-cnibin\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:39.956435 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956341 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-etc-selinux\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:39.956435 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956355 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-sys-fs\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:39.956435 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956368 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-kubernetes\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:39.956435 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956386 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-cni-dir\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:39.956435 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956410 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-slash\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.956435 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956431 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-var-lib-openvswitch\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.956824 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956454 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-env-overrides\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.956824 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956494 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqsv7\" (UniqueName: \"kubernetes.io/projected/7eed8239-b695-41f7-968f-b3a02659b44c-kube-api-access-wqsv7\") pod \"iptables-alerter-lnjwb\" (UID: \"7eed8239-b695-41f7-968f-b3a02659b44c\") " pod="openshift-network-operator/iptables-alerter-lnjwb" Oct 30 18:37:39.956824 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956511 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-ovnkube-config\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.956824 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956525 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bfeb3c6-d71a-4b5e-96db-198da488f7a9-tmp-dir\") pod \"node-resolver-5mmnd\" (UID: \"2bfeb3c6-d71a-4b5e-96db-198da488f7a9\") " pod="openshift-dns/node-resolver-5mmnd" Oct 30 18:37:39.956824 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956545 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c89d1725-2dbb-49dc-92b1-218935ce8a74-cni-binary-copy\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:39.956824 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956566 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9hs4\" (UniqueName: \"kubernetes.io/projected/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-kube-api-access-b9hs4\") pod \"network-metrics-daemon-2s5rm\" (UID: \"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2\") " pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:39.956824 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956599 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6tt86\" (UniqueName: \"kubernetes.io/projected/15f5b642-cd06-42ad-9832-dcfe438ea4e5-kube-api-access-6tt86\") pod \"node-ca-kjn4r\" (UID: \"15f5b642-cd06-42ad-9832-dcfe438ea4e5\") " pod="openshift-image-registry/node-ca-kjn4r" Oct 30 18:37:39.956824 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956626 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-run-netns\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.956824 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956648 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-cni-bin\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.956824 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956663 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7eed8239-b695-41f7-968f-b3a02659b44c-iptables-alerter-script\") pod \"iptables-alerter-lnjwb\" (UID: \"7eed8239-b695-41f7-968f-b3a02659b44c\") " pod="openshift-network-operator/iptables-alerter-lnjwb" Oct 30 18:37:39.956824 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956691 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-host\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:39.956824 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956712 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qndwx\" (UniqueName: \"kubernetes.io/projected/2bfeb3c6-d71a-4b5e-96db-198da488f7a9-kube-api-access-qndwx\") pod \"node-resolver-5mmnd\" (UID: \"2bfeb3c6-d71a-4b5e-96db-198da488f7a9\") " pod="openshift-dns/node-resolver-5mmnd" Oct 30 18:37:39.956824 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956726 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8c9b04e2-b099-4704-8786-b64a8991d0f4-dbus\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:39.956824 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956739 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-var-lib-kubelet\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:39.956824 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956753 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-hostroot\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:39.956824 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956772 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-daemon-config\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:39.957267 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956805 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-ovn-node-metrics-cert\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.957267 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956821 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b9a31e1-73c6-4617-ac72-296e03e928f2-agent-certs\") pod \"konnectivity-agent-bchfv\" (UID: \"4b9a31e1-73c6-4617-ac72-296e03e928f2\") " pod="kube-system/konnectivity-agent-bchfv" Oct 30 18:37:39.957267 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956839 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b9a31e1-73c6-4617-ac72-296e03e928f2-konnectivity-ca\") pod \"konnectivity-agent-bchfv\" (UID: \"4b9a31e1-73c6-4617-ac72-296e03e928f2\") " pod="kube-system/konnectivity-agent-bchfv" Oct 30 18:37:39.957267 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956853 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:39.957267 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956867 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjh2m\" (UniqueName: \"kubernetes.io/projected/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-kube-api-access-qjh2m\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:39.957267 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956882 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-etc-openvswitch\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.957267 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956902 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmb9d\" (UniqueName: \"kubernetes.io/projected/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-kube-api-access-xmb9d\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:39.957267 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.956916 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:39.990457 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.990438 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.990556 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.990470 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 18:37:39.990556 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.990501 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.990556 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.990518 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 18:37:39.990556 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.990526 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.990556 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.990508 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.990872 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.990855 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-q8mbb\"" Oct 30 18:37:39.992090 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.992076 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.992155 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.992116 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 18:37:39.992201 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.992167 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 18:37:39.992201 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.992167 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-m9qwf\"" Oct 30 18:37:39.995312 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.995300 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 18:37:39.995354 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.995312 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h468w\"" Oct 30 18:37:39.995354 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.995314 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.996424 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.996413 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 18:37:39.996500 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.996488 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kbv4s\"" Oct 30 18:37:39.996543 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.996522 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 18:37:39.996600 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.996583 2578 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Oct 30 18:37:39.996600 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.996597 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gk56f\"" Oct 30 18:37:39.996668 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:39.996634 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-dh4rs" Oct 30 18:37:40.057771 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.057748 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztsmk\" (UniqueName: \"kubernetes.io/projected/e2455903-bb15-41e2-8d91-51df8775b033-kube-api-access-ztsmk\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:40.057873 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.057772 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-modprobe-d\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.057873 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.057792 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-cnibin\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.057873 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.057806 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-socket-dir-parent\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.057963 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.057896 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-cnibin\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.057963 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.057915 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-modprobe-d\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.057963 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.057924 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-socket-dir-parent\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.057963 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.057951 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-var-lib-cni-bin\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.058077 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.057969 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-var-lib-cni-multus\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.058077 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.057987 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-var-lib-cni-multus\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.058077 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058016 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-var-lib-cni-bin\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.058159 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058074 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-run-multus-certs\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.058159 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058104 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkqwc\" (UniqueName: \"kubernetes.io/projected/c89d1725-2dbb-49dc-92b1-218935ce8a74-kube-api-access-fkqwc\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.058159 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058137 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-run-multus-certs\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.058159 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058142 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-run-ovn\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.058265 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058168 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-run-netns\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.058265 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058205 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-run-ovn\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.058265 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058206 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-run-ovn-kubernetes\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.058345 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058272 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-run-netns\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.058345 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058285 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-cni-netd\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.058345 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058327 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-cni-netd\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.058345 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058306 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-run-ovn-kubernetes\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.058345 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058333 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs\") pod \"network-metrics-daemon-2s5rm\" (UID: \"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2\") " pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:40.058508 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058371 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7eed8239-b695-41f7-968f-b3a02659b44c-host-slash\") pod \"iptables-alerter-lnjwb\" (UID: \"7eed8239-b695-41f7-968f-b3a02659b44c\") " pod="openshift-network-operator/iptables-alerter-lnjwb" Oct 30 18:37:40.058508 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058387 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:40.058508 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058401 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-device-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:40.058508 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:40.058407 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:40.058508 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058415 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-system-cni-dir\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.058508 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058436 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-run-systemd\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.058508 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058467 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:40.058508 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:40.058479 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs podName:8d3a00e6-202e-4ffb-b8e0-67f15d831ae2 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:40.558446057 +0000 UTC m=+2.272047367 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs") pod "network-metrics-daemon-2s5rm" (UID: "8d3a00e6-202e-4ffb-b8e0-67f15d831ae2") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:40.058508 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058489 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-system-cni-dir\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.058508 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058496 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-log-socket\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.058508 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058502 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7eed8239-b695-41f7-968f-b3a02659b44c-host-slash\") pod \"iptables-alerter-lnjwb\" (UID: \"7eed8239-b695-41f7-968f-b3a02659b44c\") " pod="openshift-network-operator/iptables-alerter-lnjwb" Oct 30 18:37:40.058889 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058514 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-run-systemd\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.058889 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058520 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-sysconfig\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.058889 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058512 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-device-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:40.058889 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058654 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-log-socket\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.058889 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058698 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-sysconfig\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.059035 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058979 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-sys\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.059035 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.058999 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-var-lib-kubelet\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.059035 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059016 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-cnibin\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:40.059035 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059023 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-sys\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.059035 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059029 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-etc-selinux\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:40.059202 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059056 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-sys-fs\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:40.059202 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059084 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-kubernetes\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.059202 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059095 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-cnibin\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:40.059202 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059102 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-var-lib-kubelet\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.059202 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059111 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-etc-selinux\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:40.059202 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059106 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-sys-fs\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:40.059202 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059127 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-kubernetes\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.059202 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059109 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-cni-dir\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.059202 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059150 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-slash\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.059202 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059165 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-var-lib-openvswitch\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.059202 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059181 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-env-overrides\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.059202 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059195 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqsv7\" (UniqueName: \"kubernetes.io/projected/7eed8239-b695-41f7-968f-b3a02659b44c-kube-api-access-wqsv7\") pod \"iptables-alerter-lnjwb\" (UID: \"7eed8239-b695-41f7-968f-b3a02659b44c\") " pod="openshift-network-operator/iptables-alerter-lnjwb" Oct 30 18:37:40.059766 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059208 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-cni-dir\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.059766 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059214 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-ovnkube-config\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.059766 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059211 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-slash\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.059766 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059241 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bfeb3c6-d71a-4b5e-96db-198da488f7a9-tmp-dir\") pod \"node-resolver-5mmnd\" (UID: \"2bfeb3c6-d71a-4b5e-96db-198da488f7a9\") " pod="openshift-dns/node-resolver-5mmnd" Oct 30 18:37:40.059766 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059252 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-var-lib-openvswitch\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.059766 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059258 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c89d1725-2dbb-49dc-92b1-218935ce8a74-cni-binary-copy\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.059766 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059312 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9hs4\" (UniqueName: \"kubernetes.io/projected/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-kube-api-access-b9hs4\") pod \"network-metrics-daemon-2s5rm\" (UID: \"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2\") " pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:40.059766 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059341 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tt86\" (UniqueName: \"kubernetes.io/projected/15f5b642-cd06-42ad-9832-dcfe438ea4e5-kube-api-access-6tt86\") pod \"node-ca-kjn4r\" (UID: \"15f5b642-cd06-42ad-9832-dcfe438ea4e5\") " pod="openshift-image-registry/node-ca-kjn4r" Oct 30 18:37:40.059766 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059368 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-run-netns\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.059766 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059391 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-cni-bin\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.059766 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059414 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7eed8239-b695-41f7-968f-b3a02659b44c-iptables-alerter-script\") pod \"iptables-alerter-lnjwb\" (UID: \"7eed8239-b695-41f7-968f-b3a02659b44c\") " pod="openshift-network-operator/iptables-alerter-lnjwb" Oct 30 18:37:40.059766 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059436 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-host\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.059766 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059459 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qndwx\" (UniqueName: \"kubernetes.io/projected/2bfeb3c6-d71a-4b5e-96db-198da488f7a9-kube-api-access-qndwx\") pod \"node-resolver-5mmnd\" (UID: \"2bfeb3c6-d71a-4b5e-96db-198da488f7a9\") " pod="openshift-dns/node-resolver-5mmnd" Oct 30 18:37:40.059766 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059482 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8c9b04e2-b099-4704-8786-b64a8991d0f4-dbus\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:40.059766 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059506 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-var-lib-kubelet\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.059766 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059530 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-hostroot\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.059766 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059554 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-daemon-config\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.059766 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059557 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bfeb3c6-d71a-4b5e-96db-198da488f7a9-tmp-dir\") pod \"node-resolver-5mmnd\" (UID: \"2bfeb3c6-d71a-4b5e-96db-198da488f7a9\") " pod="openshift-dns/node-resolver-5mmnd" Oct 30 18:37:40.060617 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059604 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-cni-bin\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.060617 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059618 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-ovn-node-metrics-cert\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.060617 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059646 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b9a31e1-73c6-4617-ac72-296e03e928f2-agent-certs\") pod \"konnectivity-agent-bchfv\" (UID: \"4b9a31e1-73c6-4617-ac72-296e03e928f2\") " pod="kube-system/konnectivity-agent-bchfv" Oct 30 18:37:40.060617 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059670 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b9a31e1-73c6-4617-ac72-296e03e928f2-konnectivity-ca\") pod \"konnectivity-agent-bchfv\" (UID: \"4b9a31e1-73c6-4617-ac72-296e03e928f2\") " pod="kube-system/konnectivity-agent-bchfv" Oct 30 18:37:40.060617 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059671 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-var-lib-kubelet\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.060617 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059698 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8c9b04e2-b099-4704-8786-b64a8991d0f4-dbus\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:40.060617 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059707 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:40.060617 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059730 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-hostroot\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.060617 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059734 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjh2m\" (UniqueName: \"kubernetes.io/projected/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-kube-api-access-qjh2m\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:40.060617 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059747 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-host\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.060617 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059759 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-etc-openvswitch\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.060617 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059790 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmb9d\" (UniqueName: \"kubernetes.io/projected/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-kube-api-access-xmb9d\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.060617 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059819 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:40.060617 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059840 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/15f5b642-cd06-42ad-9832-dcfe438ea4e5-serviceca\") pod \"node-ca-kjn4r\" (UID: \"15f5b642-cd06-42ad-9832-dcfe438ea4e5\") " pod="openshift-image-registry/node-ca-kjn4r" Oct 30 18:37:40.060617 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059873 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l7k8\" (UniqueName: \"kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8\") pod \"network-check-target-95g7n\" (UID: \"cd195fdf-8022-4279-8eca-15d145d59038\") " pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:37:40.060617 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059900 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8c9b04e2-b099-4704-8786-b64a8991d0f4-kubelet-config\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:40.060617 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059761 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-env-overrides\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.061323 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059925 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-tuned\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.061323 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059940 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:40.061323 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059955 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thmjv\" (UniqueName: \"kubernetes.io/projected/6b091344-3cff-4f72-9512-02499eb9a9ef-kube-api-access-thmjv\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.061323 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059979 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-etc-kubernetes\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.061323 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059985 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-run-netns\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.061323 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059898 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c89d1725-2dbb-49dc-92b1-218935ce8a74-cni-binary-copy\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.061323 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059999 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.061323 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060021 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-run\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.061323 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060042 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2bfeb3c6-d71a-4b5e-96db-198da488f7a9-hosts-file\") pod \"node-resolver-5mmnd\" (UID: \"2bfeb3c6-d71a-4b5e-96db-198da488f7a9\") " pod="openshift-dns/node-resolver-5mmnd" Oct 30 18:37:40.061323 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060064 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15f5b642-cd06-42ad-9832-dcfe438ea4e5-host\") pod \"node-ca-kjn4r\" (UID: \"15f5b642-cd06-42ad-9832-dcfe438ea4e5\") " pod="openshift-image-registry/node-ca-kjn4r" Oct 30 18:37:40.061323 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060087 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-kubelet\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.061323 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.059900 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-ovnkube-config\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.061323 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060111 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-registration-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:40.061323 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060137 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-lib-modules\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.061323 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060162 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-system-cni-dir\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:40.061323 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060185 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:40.061323 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060164 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.062048 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060238 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-run-openvswitch\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.062048 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060265 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-sysctl-d\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.062048 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060259 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-etc-openvswitch\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.062048 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060290 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:40.062048 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060300 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8c9b04e2-b099-4704-8786-b64a8991d0f4-kubelet-config\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:40.062048 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060297 2578 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 18:37:40.062048 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060366 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-run-k8s-cni-cncf-io\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.062048 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060407 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-run\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.062048 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060437 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2bfeb3c6-d71a-4b5e-96db-198da488f7a9-hosts-file\") pod \"node-resolver-5mmnd\" (UID: \"2bfeb3c6-d71a-4b5e-96db-198da488f7a9\") " pod="openshift-dns/node-resolver-5mmnd" Oct 30 18:37:40.062048 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060441 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-etc-kubernetes\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.062048 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060462 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15f5b642-cd06-42ad-9832-dcfe438ea4e5-host\") pod \"node-ca-kjn4r\" (UID: \"15f5b642-cd06-42ad-9832-dcfe438ea4e5\") " pod="openshift-image-registry/node-ca-kjn4r" Oct 30 18:37:40.062048 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060473 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-system-cni-dir\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:40.062048 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060512 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-kubelet\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.062048 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060608 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-registration-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:40.062048 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060319 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-run-k8s-cni-cncf-io\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.062048 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060654 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-conf-dir\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.062048 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060669 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-node-log\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.062048 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060682 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-lib-modules\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.062551 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060707 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-ovnkube-script-lib\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.062551 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060747 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-socket-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:40.062551 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060761 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b091344-3cff-4f72-9512-02499eb9a9ef-tmp\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.062551 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060791 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-sysctl-d\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.062551 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060794 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-node-log\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.062551 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:40.060822 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:40.062551 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060827 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-systemd-units\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.062551 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060851 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-cni-binary-copy\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:40.062551 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060843 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-run-openvswitch\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.062551 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:40.060865 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret podName:8c9b04e2-b099-4704-8786-b64a8991d0f4 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:40.560849546 +0000 UTC m=+2.274450866 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret") pod "global-pull-secret-syncer-hmzzs" (UID: "8c9b04e2-b099-4704-8786-b64a8991d0f4") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:40.062551 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060898 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-sysctl-conf\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.062551 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060925 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-systemd\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.062551 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060970 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-systemd\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.062551 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060990 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-conf-dir\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.062551 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.060996 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-systemd-units\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.062551 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.061111 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-sysctl-conf\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.062551 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.061112 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-socket-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:40.063160 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.061141 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b9a31e1-73c6-4617-ac72-296e03e928f2-konnectivity-ca\") pod \"konnectivity-agent-bchfv\" (UID: \"4b9a31e1-73c6-4617-ac72-296e03e928f2\") " pod="kube-system/konnectivity-agent-bchfv" Oct 30 18:37:40.063160 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.061288 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:40.063160 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.061653 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/15f5b642-cd06-42ad-9832-dcfe438ea4e5-serviceca\") pod \"node-ca-kjn4r\" (UID: \"15f5b642-cd06-42ad-9832-dcfe438ea4e5\") " pod="openshift-image-registry/node-ca-kjn4r" Oct 30 18:37:40.063160 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.061658 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-daemon-config\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.063160 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.062927 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-ovnkube-script-lib\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.063160 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.062933 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:40.063160 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.063052 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-cni-binary-copy\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:40.063160 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.063124 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7eed8239-b695-41f7-968f-b3a02659b44c-iptables-alerter-script\") pod \"iptables-alerter-lnjwb\" (UID: \"7eed8239-b695-41f7-968f-b3a02659b44c\") " pod="openshift-network-operator/iptables-alerter-lnjwb" Oct 30 18:37:40.063835 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.063815 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b091344-3cff-4f72-9512-02499eb9a9ef-tmp\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.063916 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.063845 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-tuned\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.063916 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.063898 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-ovn-node-metrics-cert\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.064060 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.064041 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b9a31e1-73c6-4617-ac72-296e03e928f2-agent-certs\") pod \"konnectivity-agent-bchfv\" (UID: \"4b9a31e1-73c6-4617-ac72-296e03e928f2\") " pod="kube-system/konnectivity-agent-bchfv" Oct 30 18:37:40.114311 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:40.114256 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:37:40.114311 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:40.114277 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:37:40.114311 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:40.114289 2578 projected.go:196] Error preparing data for projected volume kube-api-access-6l7k8 for pod openshift-network-diagnostics/network-check-target-95g7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:40.114486 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:40.114347 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8 podName:cd195fdf-8022-4279-8eca-15d145d59038 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:40.614330607 +0000 UTC m=+2.327931921 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-6l7k8" (UniqueName: "kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8") pod "network-check-target-95g7n" (UID: "cd195fdf-8022-4279-8eca-15d145d59038") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:40.116504 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.116459 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztsmk\" (UniqueName: \"kubernetes.io/projected/e2455903-bb15-41e2-8d91-51df8775b033-kube-api-access-ztsmk\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:40.117216 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.117185 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkqwc\" (UniqueName: \"kubernetes.io/projected/c89d1725-2dbb-49dc-92b1-218935ce8a74-kube-api-access-fkqwc\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.117312 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.117226 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqsv7\" (UniqueName: \"kubernetes.io/projected/7eed8239-b695-41f7-968f-b3a02659b44c-kube-api-access-wqsv7\") pod \"iptables-alerter-lnjwb\" (UID: \"7eed8239-b695-41f7-968f-b3a02659b44c\") " pod="openshift-network-operator/iptables-alerter-lnjwb" Oct 30 18:37:40.120242 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.120215 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjh2m\" (UniqueName: \"kubernetes.io/projected/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-kube-api-access-qjh2m\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:40.120364 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.120257 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9hs4\" (UniqueName: \"kubernetes.io/projected/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-kube-api-access-b9hs4\") pod \"network-metrics-daemon-2s5rm\" (UID: \"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2\") " pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:40.120364 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.120281 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tt86\" (UniqueName: \"kubernetes.io/projected/15f5b642-cd06-42ad-9832-dcfe438ea4e5-kube-api-access-6tt86\") pod \"node-ca-kjn4r\" (UID: \"15f5b642-cd06-42ad-9832-dcfe438ea4e5\") " pod="openshift-image-registry/node-ca-kjn4r" Oct 30 18:37:40.121051 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.121031 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qndwx\" (UniqueName: \"kubernetes.io/projected/2bfeb3c6-d71a-4b5e-96db-198da488f7a9-kube-api-access-qndwx\") pod \"node-resolver-5mmnd\" (UID: \"2bfeb3c6-d71a-4b5e-96db-198da488f7a9\") " pod="openshift-dns/node-resolver-5mmnd" Oct 30 18:37:40.123062 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.123042 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmb9d\" (UniqueName: \"kubernetes.io/projected/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-kube-api-access-xmb9d\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.123140 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.123128 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thmjv\" (UniqueName: \"kubernetes.io/projected/6b091344-3cff-4f72-9512-02499eb9a9ef-kube-api-access-thmjv\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.215156 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.215120 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-m5xnp" Oct 30 18:37:40.235253 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.235227 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-lnjwb" Oct 30 18:37:40.256952 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.256927 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:37:40.257047 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.256983 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bchfv" Oct 30 18:37:40.274409 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.274383 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:37:40.276526 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.276505 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:37:40.303238 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:40.303209 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b091344_3cff_4f72_9512_02499eb9a9ef.slice/crio-5bee4fa2d5b0160c58fc68dab7f8a2037268ed6cd8e13e5cc32df820d8c66b6a WatchSource:0}: Error finding container 5bee4fa2d5b0160c58fc68dab7f8a2037268ed6cd8e13e5cc32df820d8c66b6a: Status 404 returned error can't find the container with id 5bee4fa2d5b0160c58fc68dab7f8a2037268ed6cd8e13e5cc32df820d8c66b6a Oct 30 18:37:40.303696 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:40.303669 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2455903_bb15_41e2_8d91_51df8775b033.slice/crio-f7b2298ae1b4cf4ef2cbb4e235848ecc9fdfe1cbcb06169900783b1ecdec7e47 WatchSource:0}: Error finding container f7b2298ae1b4cf4ef2cbb4e235848ecc9fdfe1cbcb06169900783b1ecdec7e47: Status 404 returned error can't find the container with id f7b2298ae1b4cf4ef2cbb4e235848ecc9fdfe1cbcb06169900783b1ecdec7e47 Oct 30 18:37:40.304474 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:40.304454 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b9a31e1_73c6_4617_ac72_296e03e928f2.slice/crio-2fd69b6a889132d82f4380a7c8a7a16109a844804e2c03a418e94af8a666f6bf WatchSource:0}: Error finding container 2fd69b6a889132d82f4380a7c8a7a16109a844804e2c03a418e94af8a666f6bf: Status 404 returned error can't find the container with id 2fd69b6a889132d82f4380a7c8a7a16109a844804e2c03a418e94af8a666f6bf Oct 30 18:37:40.305383 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:40.305363 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod99a39564afa8d640f032873b7cd4dd97.slice/crio-e035e6d046c09feaf39dad9de2fed20fa7f62fac403a143af5925da680b22ca2 WatchSource:0}: Error finding container e035e6d046c09feaf39dad9de2fed20fa7f62fac403a143af5925da680b22ca2: Status 404 returned error can't find the container with id e035e6d046c09feaf39dad9de2fed20fa7f62fac403a143af5925da680b22ca2 Oct 30 18:37:40.306031 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:40.306015 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dbda9e04d721f174c9b45daae509010.slice/crio-1def0758feaa34df841ee35ec0c9744f1652ec4e3db9d160b8212fff4fb5ed04 WatchSource:0}: Error finding container 1def0758feaa34df841ee35ec0c9744f1652ec4e3db9d160b8212fff4fb5ed04: Status 404 returned error can't find the container with id 1def0758feaa34df841ee35ec0c9744f1652ec4e3db9d160b8212fff4fb5ed04 Oct 30 18:37:40.308531 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.308515 2578 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:37:40.311078 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.311064 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:37:40.312899 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.312883 2578 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:37:40.314036 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.314020 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:37:40.320899 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.320883 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5mmnd" Oct 30 18:37:40.323295 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:40.323274 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44c58e2f_bf8e_41d1_9bb2_be8fc0e2eb00.slice/crio-ccf8d80ee7794e49416a9b3d06822143ece1c2af772de7d1b76b0faa5bbd58d5 WatchSource:0}: Error finding container ccf8d80ee7794e49416a9b3d06822143ece1c2af772de7d1b76b0faa5bbd58d5: Status 404 returned error can't find the container with id ccf8d80ee7794e49416a9b3d06822143ece1c2af772de7d1b76b0faa5bbd58d5 Oct 30 18:37:40.325203 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.325184 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-kjn4r" Oct 30 18:37:40.329494 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:40.329476 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bfeb3c6_d71a_4b5e_96db_198da488f7a9.slice/crio-234c59b104002e47f821c4f580fd993ad867affa8bd4ad0a833ceb243b979f03 WatchSource:0}: Error finding container 234c59b104002e47f821c4f580fd993ad867affa8bd4ad0a833ceb243b979f03: Status 404 returned error can't find the container with id 234c59b104002e47f821c4f580fd993ad867affa8bd4ad0a833ceb243b979f03 Oct 30 18:37:40.334090 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:40.334072 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod15f5b642_cd06_42ad_9832_dcfe438ea4e5.slice/crio-4656cde633a452376680ea5749895c5b4884b3df064fd06daafcf44e6f1ff605 WatchSource:0}: Error finding container 4656cde633a452376680ea5749895c5b4884b3df064fd06daafcf44e6f1ff605: Status 404 returned error can't find the container with id 4656cde633a452376680ea5749895c5b4884b3df064fd06daafcf44e6f1ff605 Oct 30 18:37:40.563796 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.563720 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:40.563796 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.563759 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs\") pod \"network-metrics-daemon-2s5rm\" (UID: \"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2\") " pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:40.563964 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:40.563848 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:40.563964 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:40.563894 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs podName:8d3a00e6-202e-4ffb-b8e0-67f15d831ae2 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:41.563880775 +0000 UTC m=+3.277482071 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs") pod "network-metrics-daemon-2s5rm" (UID: "8d3a00e6-202e-4ffb-b8e0-67f15d831ae2") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:40.563964 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:40.563849 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:40.563964 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:40.563922 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret podName:8c9b04e2-b099-4704-8786-b64a8991d0f4 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:41.56391589 +0000 UTC m=+3.277517187 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret") pod "global-pull-secret-syncer-hmzzs" (UID: "8c9b04e2-b099-4704-8786-b64a8991d0f4") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:40.664731 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.664698 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l7k8\" (UniqueName: \"kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8\") pod \"network-check-target-95g7n\" (UID: \"cd195fdf-8022-4279-8eca-15d145d59038\") " pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:37:40.665333 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:40.664829 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:37:40.665333 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:40.664842 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:37:40.665333 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:40.664851 2578 projected.go:196] Error preparing data for projected volume kube-api-access-6l7k8 for pod openshift-network-diagnostics/network-check-target-95g7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:40.665333 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:40.664900 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8 podName:cd195fdf-8022-4279-8eca-15d145d59038 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:41.664884621 +0000 UTC m=+3.378485928 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-6l7k8" (UniqueName: "kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8") pod "network-check-target-95g7n" (UID: "cd195fdf-8022-4279-8eca-15d145d59038") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:40.718564 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.718512 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" event={"ID":"6b091344-3cff-4f72-9512-02499eb9a9ef","Type":"ContainerStarted","Data":"5bee4fa2d5b0160c58fc68dab7f8a2037268ed6cd8e13e5cc32df820d8c66b6a"} Oct 30 18:37:40.719487 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.719468 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal" event={"ID":"99a39564afa8d640f032873b7cd4dd97","Type":"ContainerStarted","Data":"e035e6d046c09feaf39dad9de2fed20fa7f62fac403a143af5925da680b22ca2"} Oct 30 18:37:40.720488 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.720459 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" event={"ID":"e2455903-bb15-41e2-8d91-51df8775b033","Type":"ContainerStarted","Data":"f7b2298ae1b4cf4ef2cbb4e235848ecc9fdfe1cbcb06169900783b1ecdec7e47"} Oct 30 18:37:40.721423 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.721405 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bchfv" event={"ID":"4b9a31e1-73c6-4617-ac72-296e03e928f2","Type":"ContainerStarted","Data":"2fd69b6a889132d82f4380a7c8a7a16109a844804e2c03a418e94af8a666f6bf"} Oct 30 18:37:40.722417 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.722395 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kjn4r" event={"ID":"15f5b642-cd06-42ad-9832-dcfe438ea4e5","Type":"ContainerStarted","Data":"4656cde633a452376680ea5749895c5b4884b3df064fd06daafcf44e6f1ff605"} Oct 30 18:37:40.723422 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.723404 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5mmnd" event={"ID":"2bfeb3c6-d71a-4b5e-96db-198da488f7a9","Type":"ContainerStarted","Data":"234c59b104002e47f821c4f580fd993ad867affa8bd4ad0a833ceb243b979f03"} Oct 30 18:37:40.724532 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.724514 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bh8c5" event={"ID":"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00","Type":"ContainerStarted","Data":"ccf8d80ee7794e49416a9b3d06822143ece1c2af772de7d1b76b0faa5bbd58d5"} Oct 30 18:37:40.725593 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.725564 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" event={"ID":"7dbda9e04d721f174c9b45daae509010","Type":"ContainerStarted","Data":"1def0758feaa34df841ee35ec0c9744f1652ec4e3db9d160b8212fff4fb5ed04"} Oct 30 18:37:40.999631 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.999535 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:32:39 +0000 UTC" deadline="2028-04-22 14:50:30.512619646 +0000 UTC" Oct 30 18:37:40.999631 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:40.999586 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12956h12m49.513036295s" Oct 30 18:37:41.059552 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:41.059514 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7eed8239_b695_41f7_968f_b3a02659b44c.slice/crio-510a7c62896ac8e28e4f9b7c8a70f158b31b85c7f39b6faae1447460f7418a12 WatchSource:0}: Error finding container 510a7c62896ac8e28e4f9b7c8a70f158b31b85c7f39b6faae1447460f7418a12: Status 404 returned error can't find the container with id 510a7c62896ac8e28e4f9b7c8a70f158b31b85c7f39b6faae1447460f7418a12 Oct 30 18:37:41.071967 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:41.071944 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc89d1725_2dbb_49dc_92b1_218935ce8a74.slice/crio-a3d0c2abffcf4a28f3b6343dea6d745d27fd48780411b32639a996672e47416d WatchSource:0}: Error finding container a3d0c2abffcf4a28f3b6343dea6d745d27fd48780411b32639a996672e47416d: Status 404 returned error can't find the container with id a3d0c2abffcf4a28f3b6343dea6d745d27fd48780411b32639a996672e47416d Oct 30 18:37:41.198473 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:37:41.198390 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce6f0c54_70e4_4d45_8bc3_2823a815f0db.slice/crio-a590889638ad55563f8b4a4f6a2914ff008c4675b8cc971a553efbb73101bc21 WatchSource:0}: Error finding container a590889638ad55563f8b4a4f6a2914ff008c4675b8cc971a553efbb73101bc21: Status 404 returned error can't find the container with id a590889638ad55563f8b4a4f6a2914ff008c4675b8cc971a553efbb73101bc21 Oct 30 18:37:41.571143 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:41.571083 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:41.571143 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:41.571148 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs\") pod \"network-metrics-daemon-2s5rm\" (UID: \"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2\") " pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:41.571389 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:41.571281 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:41.571389 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:41.571342 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs podName:8d3a00e6-202e-4ffb-b8e0-67f15d831ae2 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:43.571323433 +0000 UTC m=+5.284924729 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs") pod "network-metrics-daemon-2s5rm" (UID: "8d3a00e6-202e-4ffb-b8e0-67f15d831ae2") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:41.571501 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:41.571404 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:41.571501 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:41.571440 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret podName:8c9b04e2-b099-4704-8786-b64a8991d0f4 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:43.57142866 +0000 UTC m=+5.285029953 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret") pod "global-pull-secret-syncer-hmzzs" (UID: "8c9b04e2-b099-4704-8786-b64a8991d0f4") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:41.671739 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:41.671700 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l7k8\" (UniqueName: \"kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8\") pod \"network-check-target-95g7n\" (UID: \"cd195fdf-8022-4279-8eca-15d145d59038\") " pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:37:41.672281 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:41.671859 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:37:41.672281 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:41.671878 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:37:41.672281 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:41.671889 2578 projected.go:196] Error preparing data for projected volume kube-api-access-6l7k8 for pod openshift-network-diagnostics/network-check-target-95g7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:41.672281 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:41.671946 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8 podName:cd195fdf-8022-4279-8eca-15d145d59038 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:43.671929458 +0000 UTC m=+5.385530756 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-6l7k8" (UniqueName: "kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8") pod "network-check-target-95g7n" (UID: "cd195fdf-8022-4279-8eca-15d145d59038") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:41.715544 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:41.715514 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:41.715710 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:41.715665 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2s5rm" podUID="8d3a00e6-202e-4ffb-b8e0-67f15d831ae2" Oct 30 18:37:41.715782 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:41.715744 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:37:41.715834 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:41.715815 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95g7n" podUID="cd195fdf-8022-4279-8eca-15d145d59038" Oct 30 18:37:41.715887 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:41.715874 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:41.715960 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:41.715942 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hmzzs" podUID="8c9b04e2-b099-4704-8786-b64a8991d0f4" Oct 30 18:37:41.736604 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:41.736554 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lnjwb" event={"ID":"7eed8239-b695-41f7-968f-b3a02659b44c","Type":"ContainerStarted","Data":"510a7c62896ac8e28e4f9b7c8a70f158b31b85c7f39b6faae1447460f7418a12"} Oct 30 18:37:41.738285 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:41.738257 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" event={"ID":"ce6f0c54-70e4-4d45-8bc3-2823a815f0db","Type":"ContainerStarted","Data":"a590889638ad55563f8b4a4f6a2914ff008c4675b8cc971a553efbb73101bc21"} Oct 30 18:37:41.744156 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:41.744132 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m5xnp" event={"ID":"c89d1725-2dbb-49dc-92b1-218935ce8a74","Type":"ContainerStarted","Data":"a3d0c2abffcf4a28f3b6343dea6d745d27fd48780411b32639a996672e47416d"} Oct 30 18:37:42.000419 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:42.000312 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:32:39 +0000 UTC" deadline="2028-05-25 02:59:06.528697252 +0000 UTC" Oct 30 18:37:42.000419 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:42.000377 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13736h21m24.528324737s" Oct 30 18:37:43.590493 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:43.590412 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:43.590493 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:43.590474 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs\") pod \"network-metrics-daemon-2s5rm\" (UID: \"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2\") " pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:43.591305 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:43.590657 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:43.591305 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:43.590723 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs podName:8d3a00e6-202e-4ffb-b8e0-67f15d831ae2 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:47.590704868 +0000 UTC m=+9.304306171 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs") pod "network-metrics-daemon-2s5rm" (UID: "8d3a00e6-202e-4ffb-b8e0-67f15d831ae2") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:43.591305 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:43.590778 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:43.591305 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:43.590801 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret podName:8c9b04e2-b099-4704-8786-b64a8991d0f4 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:47.590793681 +0000 UTC m=+9.304394987 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret") pod "global-pull-secret-syncer-hmzzs" (UID: "8c9b04e2-b099-4704-8786-b64a8991d0f4") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:43.691251 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:43.691220 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l7k8\" (UniqueName: \"kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8\") pod \"network-check-target-95g7n\" (UID: \"cd195fdf-8022-4279-8eca-15d145d59038\") " pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:37:43.691423 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:43.691369 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:37:43.691423 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:43.691388 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:37:43.691423 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:43.691400 2578 projected.go:196] Error preparing data for projected volume kube-api-access-6l7k8 for pod openshift-network-diagnostics/network-check-target-95g7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:43.691602 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:43.691449 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8 podName:cd195fdf-8022-4279-8eca-15d145d59038 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:47.691437029 +0000 UTC m=+9.405038338 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-6l7k8" (UniqueName: "kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8") pod "network-check-target-95g7n" (UID: "cd195fdf-8022-4279-8eca-15d145d59038") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:43.715361 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:43.715335 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:43.715512 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:43.715433 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2s5rm" podUID="8d3a00e6-202e-4ffb-b8e0-67f15d831ae2" Oct 30 18:37:43.715512 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:43.715473 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:37:43.715656 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:43.715532 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95g7n" podUID="cd195fdf-8022-4279-8eca-15d145d59038" Oct 30 18:37:43.715656 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:43.715589 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:43.715758 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:43.715660 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hmzzs" podUID="8c9b04e2-b099-4704-8786-b64a8991d0f4" Oct 30 18:37:44.787838 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:44.786021 2578 generic.go:356] "Generic (PLEG): container finished" podID="44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00" containerID="080fb3ae1d239c7bc7fd27935922348d1771c73ea19138a2e83f6725cec548d9" exitCode=0 Oct 30 18:37:44.787838 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:44.786327 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bh8c5" event={"ID":"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00","Type":"ContainerDied","Data":"080fb3ae1d239c7bc7fd27935922348d1771c73ea19138a2e83f6725cec548d9"} Oct 30 18:37:44.795379 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:44.795347 2578 generic.go:356] "Generic (PLEG): container finished" podID="7dbda9e04d721f174c9b45daae509010" containerID="d1b97879fe0f3e7c2781898547d40cc6b991e793b9fc595bbc4535ed114c2066" exitCode=0 Oct 30 18:37:44.795467 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:44.795398 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" event={"ID":"7dbda9e04d721f174c9b45daae509010","Type":"ContainerDied","Data":"d1b97879fe0f3e7c2781898547d40cc6b991e793b9fc595bbc4535ed114c2066"} Oct 30 18:37:45.715212 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:45.715122 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:45.715367 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:45.715253 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hmzzs" podUID="8c9b04e2-b099-4704-8786-b64a8991d0f4" Oct 30 18:37:45.715367 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:45.715328 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:45.715465 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:45.715409 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2s5rm" podUID="8d3a00e6-202e-4ffb-b8e0-67f15d831ae2" Oct 30 18:37:45.715513 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:45.715496 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:37:45.715597 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:45.715566 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95g7n" podUID="cd195fdf-8022-4279-8eca-15d145d59038" Oct 30 18:37:45.802378 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:45.801643 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal" event={"ID":"99a39564afa8d640f032873b7cd4dd97","Type":"ContainerStarted","Data":"896b65bf8cde3f532d8daf6aa7f29733d8d4dc48fe4b8f554450a73ad4547ea4"} Oct 30 18:37:45.805673 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:45.805635 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bchfv" event={"ID":"4b9a31e1-73c6-4617-ac72-296e03e928f2","Type":"ContainerStarted","Data":"a75b47909e42c272875229fc37e3e1dd677c000029fd67838427798db5368de6"} Oct 30 18:37:45.808888 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:45.808862 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" event={"ID":"7dbda9e04d721f174c9b45daae509010","Type":"ContainerStarted","Data":"243b12fb8743bcef79a01a360f8d101905cf6b0827ef70b5dbf787f4df1de801"} Oct 30 18:37:46.027153 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:46.026941 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal" podStartSLOduration=7.026923194 podStartE2EDuration="7.026923194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:37:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:45.913335269 +0000 UTC m=+7.626936600" watchObservedRunningTime="2026-10-30 18:37:46.026923194 +0000 UTC m=+7.740524525" Oct 30 18:37:46.140186 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:46.140131 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" podStartSLOduration=7.140111621 podStartE2EDuration="7.140111621s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:37:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:46.027496304 +0000 UTC m=+7.741097621" watchObservedRunningTime="2026-10-30 18:37:46.140111621 +0000 UTC m=+7.853712936" Oct 30 18:37:46.815454 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:46.815377 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" event={"ID":"e2455903-bb15-41e2-8d91-51df8775b033","Type":"ContainerStarted","Data":"213b8b41a8594b80ed1d91b020b17c0c696a77415dd231d9ab2f496bc4899e59"} Oct 30 18:37:46.819922 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:46.819868 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-kjn4r" event={"ID":"15f5b642-cd06-42ad-9832-dcfe438ea4e5","Type":"ContainerStarted","Data":"fb4dba62893675f006b14f8e498ada03571a1945c75f086f125ce170653849ef"} Oct 30 18:37:46.941680 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:46.941070 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bchfv" podStartSLOduration=3.627696314 podStartE2EDuration="7.940981746s" totalImagesPullingTime="4.313285432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:46.140447341 +0000 UTC m=+7.854048667" watchObservedRunningTime="2026-10-30 18:37:46.940981746 +0000 UTC m=+8.654583063" Oct 30 18:37:46.942363 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:46.942319 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-kjn4r" podStartSLOduration=2.43671302 podStartE2EDuration="7.942303243s" totalImagesPullingTime="5.505590223s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:37:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:46.940905203 +0000 UTC m=+8.654506518" watchObservedRunningTime="2026-10-30 18:37:46.942303243 +0000 UTC m=+8.655904558" Oct 30 18:37:47.622524 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:47.622417 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:47.622524 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:47.622483 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs\") pod \"network-metrics-daemon-2s5rm\" (UID: \"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2\") " pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:47.622765 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:47.622648 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:47.622765 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:47.622711 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs podName:8d3a00e6-202e-4ffb-b8e0-67f15d831ae2 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:55.622690882 +0000 UTC m=+17.336292237 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs") pod "network-metrics-daemon-2s5rm" (UID: "8d3a00e6-202e-4ffb-b8e0-67f15d831ae2") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:47.622886 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:47.622773 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:47.622886 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:47.622804 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret podName:8c9b04e2-b099-4704-8786-b64a8991d0f4 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:55.622793645 +0000 UTC m=+17.336394952 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret") pod "global-pull-secret-syncer-hmzzs" (UID: "8c9b04e2-b099-4704-8786-b64a8991d0f4") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:47.715366 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:47.715333 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:47.715630 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:47.715475 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2s5rm" podUID="8d3a00e6-202e-4ffb-b8e0-67f15d831ae2" Oct 30 18:37:47.715630 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:47.715543 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:37:47.715630 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:47.715602 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95g7n" podUID="cd195fdf-8022-4279-8eca-15d145d59038" Oct 30 18:37:47.715866 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:47.715639 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:47.715866 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:47.715702 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hmzzs" podUID="8c9b04e2-b099-4704-8786-b64a8991d0f4" Oct 30 18:37:47.723147 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:47.723118 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l7k8\" (UniqueName: \"kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8\") pod \"network-check-target-95g7n\" (UID: \"cd195fdf-8022-4279-8eca-15d145d59038\") " pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:37:47.723299 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:47.723284 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:37:47.723372 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:47.723308 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:37:47.723372 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:47.723321 2578 projected.go:196] Error preparing data for projected volume kube-api-access-6l7k8 for pod openshift-network-diagnostics/network-check-target-95g7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:47.723372 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:47.723371 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8 podName:cd195fdf-8022-4279-8eca-15d145d59038 nodeName:}" failed. No retries permitted until 2026-10-30 18:37:55.723354052 +0000 UTC m=+17.436955360 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-6l7k8" (UniqueName: "kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8") pod "network-check-target-95g7n" (UID: "cd195fdf-8022-4279-8eca-15d145d59038") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:48.730992 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:48.730969 2578 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 18:37:48.831124 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:48.831048 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" event={"ID":"e2455903-bb15-41e2-8d91-51df8775b033","Type":"ContainerStarted","Data":"b455704628f51cd0b54d67b1f1c7d7376670c48715f49fb09dafd1721bcb85eb"} Oct 30 18:37:49.708445 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:49.708330 2578 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:37:48.730989533Z","UUID":"5b88b081-80b0-4dba-bed1-e35427a44e48","Handler":null,"Name":"","Endpoint":""} Oct 30 18:37:49.713767 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:49.713736 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 18:37:49.713767 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:49.713769 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 18:37:49.714842 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:49.714813 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:49.714983 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:49.714958 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2s5rm" podUID="8d3a00e6-202e-4ffb-b8e0-67f15d831ae2" Oct 30 18:37:49.719132 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:49.719103 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:37:49.719239 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:49.719165 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:49.719722 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:49.719692 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95g7n" podUID="cd195fdf-8022-4279-8eca-15d145d59038" Oct 30 18:37:49.719851 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:49.719830 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hmzzs" podUID="8c9b04e2-b099-4704-8786-b64a8991d0f4" Oct 30 18:37:49.835115 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:49.835084 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5mmnd" event={"ID":"2bfeb3c6-d71a-4b5e-96db-198da488f7a9","Type":"ContainerStarted","Data":"50e7cd3ae59465438b16fe2b6420ab947c8703b626bdf846db951e60f0d1d43c"} Oct 30 18:37:49.838648 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:49.838622 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-m5xnp" event={"ID":"c89d1725-2dbb-49dc-92b1-218935ce8a74","Type":"ContainerStarted","Data":"f0c7f9fc533153243b0261d4cad01d436e7e9c7d3639d9f0830fb7a202a1cf47"} Oct 30 18:37:49.842239 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:49.842213 2578 generic.go:356] "Generic (PLEG): container finished" podID="44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00" containerID="055a7fa08aea0465eb32fe2b025ed798bdb47927d0a8a05b70f48ac33af074d5" exitCode=0 Oct 30 18:37:49.842358 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:49.842259 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bh8c5" event={"ID":"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00","Type":"ContainerDied","Data":"055a7fa08aea0465eb32fe2b025ed798bdb47927d0a8a05b70f48ac33af074d5"} Oct 30 18:37:49.949882 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:49.949781 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-5mmnd" podStartSLOduration=2.006284582 podStartE2EDuration="10.949764344s" totalImagesPullingTime="8.943479762s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:37:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:49.948998609 +0000 UTC m=+11.662599925" watchObservedRunningTime="2026-10-30 18:37:49.949764344 +0000 UTC m=+11.663365660" Oct 30 18:37:50.373081 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:50.373046 2578 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bchfv" Oct 30 18:37:50.374313 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:50.374288 2578 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bchfv" Oct 30 18:37:50.485836 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:50.485747 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-m5xnp" podStartSLOduration=3.399928244 podStartE2EDuration="11.485730333s" totalImagesPullingTime="8.085802089s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:37:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:50.176470513 +0000 UTC m=+11.890071825" watchObservedRunningTime="2026-10-30 18:37:50.485730333 +0000 UTC m=+12.199331648" Oct 30 18:37:50.846598 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:50.846502 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" event={"ID":"e2455903-bb15-41e2-8d91-51df8775b033","Type":"ContainerStarted","Data":"ed09a569f2c5e5fbf902084f0ed1273be340ecd583b93b89786485b5a6799b9f"} Oct 30 18:37:50.850933 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:50.850767 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-lnjwb" event={"ID":"7eed8239-b695-41f7-968f-b3a02659b44c","Type":"ContainerStarted","Data":"29ae25005c779ecdced9ed680e10d33ab5c4722f8494141c2f44526e2a224f16"} Oct 30 18:37:50.958533 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:50.958487 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" podStartSLOduration=2.451686815 podStartE2EDuration="11.958474757s" totalImagesPullingTime="9.506787942s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:37:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:50.95791646 +0000 UTC m=+12.671517774" watchObservedRunningTime="2026-10-30 18:37:50.958474757 +0000 UTC m=+12.672076107" Oct 30 18:37:51.068949 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:51.068892 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-lnjwb" podStartSLOduration=3.980404992 podStartE2EDuration="12.068876483s" totalImagesPullingTime="8.088471491s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:37:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:51.068366024 +0000 UTC m=+12.781967338" watchObservedRunningTime="2026-10-30 18:37:51.068876483 +0000 UTC m=+12.782477797" Oct 30 18:37:51.714970 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:51.714897 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:51.714970 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:51.714917 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:37:51.715144 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:51.715030 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2s5rm" podUID="8d3a00e6-202e-4ffb-b8e0-67f15d831ae2" Oct 30 18:37:51.715144 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:51.715096 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:51.715233 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:51.715170 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hmzzs" podUID="8c9b04e2-b099-4704-8786-b64a8991d0f4" Oct 30 18:37:51.715301 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:51.715273 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95g7n" podUID="cd195fdf-8022-4279-8eca-15d145d59038" Oct 30 18:37:51.855438 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:51.855403 2578 generic.go:356] "Generic (PLEG): container finished" podID="44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00" containerID="52606d8863eaeb63a2849cfaa451481b3c2aec3152be0701052bc3dffb95af6b" exitCode=0 Oct 30 18:37:51.856087 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:51.855493 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bh8c5" event={"ID":"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00","Type":"ContainerDied","Data":"52606d8863eaeb63a2849cfaa451481b3c2aec3152be0701052bc3dffb95af6b"} Oct 30 18:37:51.857329 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:51.857298 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" event={"ID":"6b091344-3cff-4f72-9512-02499eb9a9ef","Type":"ContainerStarted","Data":"58f639e8b591074667a8fe1e1027df465a37af76000228ab922a6343d3f01cd0"} Oct 30 18:37:51.864009 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:51.863987 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bchfv" Oct 30 18:37:51.864107 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:51.864086 2578 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:37:51.864721 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:51.864700 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bchfv" Oct 30 18:37:52.080378 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:52.080285 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" podStartSLOduration=1.954058028 podStartE2EDuration="13.080271798s" totalImagesPullingTime="11.12621377s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:37:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:52.08006465 +0000 UTC m=+13.793665966" watchObservedRunningTime="2026-10-30 18:37:52.080271798 +0000 UTC m=+13.793873111" Oct 30 18:37:53.715425 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:53.715227 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:53.716009 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:53.715524 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hmzzs" podUID="8c9b04e2-b099-4704-8786-b64a8991d0f4" Oct 30 18:37:53.716009 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:53.715565 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:53.716009 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:53.715670 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2s5rm" podUID="8d3a00e6-202e-4ffb-b8e0-67f15d831ae2" Oct 30 18:37:53.716009 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:53.715729 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:37:53.716009 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:53.715787 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95g7n" podUID="cd195fdf-8022-4279-8eca-15d145d59038" Oct 30 18:37:53.864675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:53.864541 2578 generic.go:356] "Generic (PLEG): container finished" podID="44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00" containerID="7689046d138217a2310d3b9ceae4f1f251559248df3dc999a1c68e581a996a33" exitCode=0 Oct 30 18:37:53.864675 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:53.864615 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bh8c5" event={"ID":"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00","Type":"ContainerDied","Data":"7689046d138217a2310d3b9ceae4f1f251559248df3dc999a1c68e581a996a33"} Oct 30 18:37:55.683624 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:55.683593 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs\") pod \"network-metrics-daemon-2s5rm\" (UID: \"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2\") " pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:55.684135 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:55.683677 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:55.684135 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:55.683769 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:55.684135 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:55.683784 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:55.684135 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:55.683840 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret podName:8c9b04e2-b099-4704-8786-b64a8991d0f4 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:11.683823234 +0000 UTC m=+33.397424527 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret") pod "global-pull-secret-syncer-hmzzs" (UID: "8c9b04e2-b099-4704-8786-b64a8991d0f4") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:37:55.684135 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:55.683859 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs podName:8d3a00e6-202e-4ffb-b8e0-67f15d831ae2 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:11.683849226 +0000 UTC m=+33.397450522 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs") pod "network-metrics-daemon-2s5rm" (UID: "8d3a00e6-202e-4ffb-b8e0-67f15d831ae2") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:37:55.714825 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:55.714793 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:37:55.714939 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:55.714921 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:55.714995 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:55.714934 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95g7n" podUID="cd195fdf-8022-4279-8eca-15d145d59038" Oct 30 18:37:55.714995 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:55.714954 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:55.715083 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:55.715043 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2s5rm" podUID="8d3a00e6-202e-4ffb-b8e0-67f15d831ae2" Oct 30 18:37:55.715137 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:55.715112 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hmzzs" podUID="8c9b04e2-b099-4704-8786-b64a8991d0f4" Oct 30 18:37:55.784428 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:55.784398 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l7k8\" (UniqueName: \"kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8\") pod \"network-check-target-95g7n\" (UID: \"cd195fdf-8022-4279-8eca-15d145d59038\") " pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:37:55.784607 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:55.784588 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Oct 30 18:37:55.784655 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:55.784617 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Oct 30 18:37:55.784655 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:55.784629 2578 projected.go:196] Error preparing data for projected volume kube-api-access-6l7k8 for pod openshift-network-diagnostics/network-check-target-95g7n: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:55.784738 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:55.784715 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8 podName:cd195fdf-8022-4279-8eca-15d145d59038 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:11.784697216 +0000 UTC m=+33.498298513 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-6l7k8" (UniqueName: "kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8") pod "network-check-target-95g7n" (UID: "cd195fdf-8022-4279-8eca-15d145d59038") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Oct 30 18:37:56.873782 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:56.873543 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" event={"ID":"ce6f0c54-70e4-4d45-8bc3-2823a815f0db","Type":"ContainerStarted","Data":"86790143c2e24f5dcad2c1c80b2f2a721cb3eaeb0d994f5fb397fb268be16127"} Oct 30 18:37:56.874143 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:56.873779 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" event={"ID":"ce6f0c54-70e4-4d45-8bc3-2823a815f0db","Type":"ContainerStarted","Data":"83caeb03878b28d20bbd74ff4f6eb57f8df3a797810983f5f308ed930cb74f3c"} Oct 30 18:37:56.874143 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:56.873834 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" event={"ID":"ce6f0c54-70e4-4d45-8bc3-2823a815f0db","Type":"ContainerStarted","Data":"ccdf36d8c8c2735c90c2fb7430aec524cac39c6cfb39926213b0d5fac10d9f71"} Oct 30 18:37:56.874143 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:56.873880 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" event={"ID":"ce6f0c54-70e4-4d45-8bc3-2823a815f0db","Type":"ContainerStarted","Data":"b24af6a69b0b244b666becf3d087f46d59d6f9c3107b8fabcc03bbc4d2b4ec96"} Oct 30 18:37:56.874143 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:56.873917 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" event={"ID":"ce6f0c54-70e4-4d45-8bc3-2823a815f0db","Type":"ContainerStarted","Data":"36752e7f4b86bfa05e463f3d0dfd1174983261af8ca59f1b76c0ca32c2698524"} Oct 30 18:37:56.874143 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:56.873950 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" event={"ID":"ce6f0c54-70e4-4d45-8bc3-2823a815f0db","Type":"ContainerStarted","Data":"89814c87847c43ad257ab1045f786977d63ce75cd618a458ca65dbb60e88692d"} Oct 30 18:37:56.876220 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:56.876192 2578 generic.go:356] "Generic (PLEG): container finished" podID="44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00" containerID="db6d97685ceccdc5eba78b0c3bb813cbcbe5a2133b6200a9f4f4857837a11354" exitCode=0 Oct 30 18:37:56.876342 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:56.876218 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bh8c5" event={"ID":"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00","Type":"ContainerDied","Data":"db6d97685ceccdc5eba78b0c3bb813cbcbe5a2133b6200a9f4f4857837a11354"} Oct 30 18:37:57.714822 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:57.714791 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:57.714973 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:57.714907 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:37:57.714973 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:57.714926 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:57.714973 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:57.714921 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2s5rm" podUID="8d3a00e6-202e-4ffb-b8e0-67f15d831ae2" Oct 30 18:37:57.715138 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:57.715008 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95g7n" podUID="cd195fdf-8022-4279-8eca-15d145d59038" Oct 30 18:37:57.715138 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:57.715088 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hmzzs" podUID="8c9b04e2-b099-4704-8786-b64a8991d0f4" Oct 30 18:37:57.880703 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:57.880672 2578 generic.go:356] "Generic (PLEG): container finished" podID="44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00" containerID="e98cb7f20d5dfbb0b5945f19f644cdfb55ac33cfc190c81f568065c24658c8f2" exitCode=0 Oct 30 18:37:57.881066 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:57.880720 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bh8c5" event={"ID":"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00","Type":"ContainerDied","Data":"e98cb7f20d5dfbb0b5945f19f644cdfb55ac33cfc190c81f568065c24658c8f2"} Oct 30 18:37:58.886143 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:58.886109 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-bh8c5" event={"ID":"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00","Type":"ContainerStarted","Data":"14fe356fee6c2b7c834fb1e3e46ce9d133854276f89f566cca4185093849f4c1"} Oct 30 18:37:58.889488 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:58.889460 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" event={"ID":"ce6f0c54-70e4-4d45-8bc3-2823a815f0db","Type":"ContainerStarted","Data":"0edc0415294b7094dcad4623308c95ea805e75a5c0463d137b8ee9b85c21eaf7"} Oct 30 18:37:59.100003 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:59.099907 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-bh8c5" podStartSLOduration=6.758560988 podStartE2EDuration="20.099893365s" totalImagesPullingTime="13.341332377s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:37:39 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:37:59.049732 +0000 UTC m=+20.763333316" watchObservedRunningTime="2026-10-30 18:37:59.099893365 +0000 UTC m=+20.813494721" Oct 30 18:37:59.715352 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:59.715320 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:37:59.715352 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:59.715343 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:37:59.715352 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:37:59.715327 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:37:59.715557 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:59.715440 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2s5rm" podUID="8d3a00e6-202e-4ffb-b8e0-67f15d831ae2" Oct 30 18:37:59.715557 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:59.715480 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hmzzs" podUID="8c9b04e2-b099-4704-8786-b64a8991d0f4" Oct 30 18:37:59.715652 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:37:59.715552 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95g7n" podUID="cd195fdf-8022-4279-8eca-15d145d59038" Oct 30 18:38:01.715607 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:01.715563 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:38:01.715607 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:01.715609 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:38:01.716340 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:01.715564 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:38:01.716340 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:01.715709 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2s5rm" podUID="8d3a00e6-202e-4ffb-b8e0-67f15d831ae2" Oct 30 18:38:01.716340 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:01.715818 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95g7n" podUID="cd195fdf-8022-4279-8eca-15d145d59038" Oct 30 18:38:01.716340 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:01.715887 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hmzzs" podUID="8c9b04e2-b099-4704-8786-b64a8991d0f4" Oct 30 18:38:01.898714 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:01.898680 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" event={"ID":"ce6f0c54-70e4-4d45-8bc3-2823a815f0db","Type":"ContainerStarted","Data":"fa9cc89ed912325e069512cf368de1a023e673a132282a998fd0ba2488720dcd"} Oct 30 18:38:01.899032 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:01.898996 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:01.899148 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:01.899066 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:01.914088 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:01.914066 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:01.916823 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:01.914547 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:02.162237 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:02.162182 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" podStartSLOduration=7.958988174 podStartE2EDuration="23.16216874s" totalImagesPullingTime="15.203180566s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:37:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:02.112301605 +0000 UTC m=+23.825902920" watchObservedRunningTime="2026-10-30 18:38:02.16216874 +0000 UTC m=+23.875770053" Oct 30 18:38:02.901270 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:02.901241 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:03.616684 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:03.616656 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hmzzs"] Oct 30 18:38:03.616850 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:03.616767 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:38:03.616898 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:03.616851 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hmzzs" podUID="8c9b04e2-b099-4704-8786-b64a8991d0f4" Oct 30 18:38:03.619503 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:03.619477 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-95g7n"] Oct 30 18:38:03.619634 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:03.619609 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:38:03.619728 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:03.619710 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95g7n" podUID="cd195fdf-8022-4279-8eca-15d145d59038" Oct 30 18:38:03.666217 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:03.666184 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2s5rm"] Oct 30 18:38:03.666376 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:03.666294 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:38:03.666414 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:03.666384 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2s5rm" podUID="8d3a00e6-202e-4ffb-b8e0-67f15d831ae2" Oct 30 18:38:04.715424 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:04.715395 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:38:04.715829 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:04.715506 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95g7n" podUID="cd195fdf-8022-4279-8eca-15d145d59038" Oct 30 18:38:05.715394 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:05.715355 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:38:05.715550 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:05.715460 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2s5rm" podUID="8d3a00e6-202e-4ffb-b8e0-67f15d831ae2" Oct 30 18:38:05.715550 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:05.715512 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:38:05.715877 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:05.715613 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hmzzs" podUID="8c9b04e2-b099-4704-8786-b64a8991d0f4" Oct 30 18:38:06.717809 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:06.717785 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:38:06.718171 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:06.717874 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95g7n" podUID="cd195fdf-8022-4279-8eca-15d145d59038" Oct 30 18:38:07.715241 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:07.715203 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:38:07.715395 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:07.715316 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2s5rm" podUID="8d3a00e6-202e-4ffb-b8e0-67f15d831ae2" Oct 30 18:38:07.715395 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:07.715367 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:38:07.715474 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:07.715454 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hmzzs" podUID="8c9b04e2-b099-4704-8786-b64a8991d0f4" Oct 30 18:38:08.716260 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:08.716224 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:38:08.716728 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:08.716323 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-95g7n" podUID="cd195fdf-8022-4279-8eca-15d145d59038" Oct 30 18:38:09.715435 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:09.715406 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:38:09.715627 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:09.715533 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-2s5rm" podUID="8d3a00e6-202e-4ffb-b8e0-67f15d831ae2" Oct 30 18:38:09.715627 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:09.715598 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:38:09.715746 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:09.715688 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-hmzzs" podUID="8c9b04e2-b099-4704-8786-b64a8991d0f4" Oct 30 18:38:10.049874 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.049794 2578 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeReady" Oct 30 18:38:10.050223 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.049911 2578 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Oct 30 18:38:10.192560 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.192532 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-pvzsh"] Oct 30 18:38:10.260089 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.260062 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-mm49z"] Oct 30 18:38:10.260223 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.260185 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-pvzsh" Oct 30 18:38:10.272920 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.272901 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-td46q"] Oct 30 18:38:10.273022 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.273010 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-mm49z" Oct 30 18:38:10.285078 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.285059 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-vwws2"] Oct 30 18:38:10.285180 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.285166 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:10.299540 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.299513 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-7c4986d449-6kk4m"] Oct 30 18:38:10.301003 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.300047 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-vwws2" Oct 30 18:38:10.312281 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.312258 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" Oct 30 18:38:10.312281 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.312273 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" Oct 30 18:38:10.321376 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.321345 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-797b6dc769-r2cdl"] Oct 30 18:38:10.321493 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.321450 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.337549 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.337531 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb"] Oct 30 18:38:10.337683 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.337665 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:10.338182 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.338163 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Oct 30 18:38:10.338281 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.338264 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 18:38:10.338426 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.338411 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 18:38:10.338589 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.338554 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Oct 30 18:38:10.338652 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.338607 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mn8z9\"" Oct 30 18:38:10.361833 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.361813 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4z4ss"] Oct 30 18:38:10.361947 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.361933 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:10.375287 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.375268 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7dbbfc9475-s4vg9"] Oct 30 18:38:10.375381 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.375369 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4z4ss" Oct 30 18:38:10.385617 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.385599 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5"] Oct 30 18:38:10.385722 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.385705 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.389259 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.389243 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-hqzbs"] Oct 30 18:38:10.389404 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.389366 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" Oct 30 18:38:10.391438 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.391416 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.391534 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.391452 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6srz4\" (UniqueName: \"kubernetes.io/projected/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-kube-api-access-6srz4\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.391644 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.391628 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-tmp\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.391706 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.391669 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2797391b-fd3d-4222-82c8-b28a2d16c9e3-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:10.391751 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.391736 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6srj\" (UniqueName: \"kubernetes.io/projected/7a4adea8-c5dc-4a4d-ad54-e985838fa6c5-kube-api-access-w6srj\") pod \"kas-connection-checker-685895c7c9-mm49z\" (UID: \"7a4adea8-c5dc-4a4d-ad54-e985838fa6c5\") " pod="kube-system/kas-connection-checker-685895c7c9-mm49z" Oct 30 18:38:10.391790 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.391778 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zfjf\" (UniqueName: \"kubernetes.io/projected/34a1d011-b30b-49b8-bc75-bfb9dbba3817-kube-api-access-4zfjf\") pod \"kas-connection-checker-685895c7c9-vwws2\" (UID: \"34a1d011-b30b-49b8-bc75-bfb9dbba3817\") " pod="kube-system/kas-connection-checker-685895c7c9-vwws2" Oct 30 18:38:10.391834 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.391801 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:10.391834 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.391820 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5w86\" (UniqueName: \"kubernetes.io/projected/a36f2ae5-c6ca-49cf-a775-82a97e92f2c4-kube-api-access-m5w86\") pod \"kas-connection-checker-685895c7c9-pvzsh\" (UID: \"a36f2ae5-c6ca-49cf-a775-82a97e92f2c4\") " pod="kube-system/kas-connection-checker-685895c7c9-pvzsh" Oct 30 18:38:10.391920 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.391906 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-snapshots\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.391987 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.391968 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lg2js\" (UniqueName: \"kubernetes.io/projected/2797391b-fd3d-4222-82c8-b28a2d16c9e3-kube-api-access-lg2js\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:10.392042 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.392001 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-service-ca-bundle\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.392042 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.392033 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-serving-cert\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.396872 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.396856 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-gg799"] Oct 30 18:38:10.397078 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.396967 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" Oct 30 18:38:10.407123 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.407103 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp"] Oct 30 18:38:10.407229 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.407215 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:10.410372 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.410356 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7f6754678-55bz6"] Oct 30 18:38:10.410493 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.410481 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" Oct 30 18:38:10.412273 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.412255 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-jwxvn"] Oct 30 18:38:10.412377 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.412363 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.413558 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.413536 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 30 18:38:10.413558 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.413538 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 30 18:38:10.413700 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.413566 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 30 18:38:10.413700 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.413667 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Oct 30 18:38:10.413798 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.413739 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 18:38:10.414391 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.414253 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:38:10.414391 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.414260 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Oct 30 18:38:10.414391 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.414385 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" Oct 30 18:38:10.414784 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.414764 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 18:38:10.414939 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.414925 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 30 18:38:10.426382 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.426302 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:10.429086 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.429070 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:38:10.434600 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.434566 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-kkzsh"] Oct 30 18:38:10.434687 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.434647 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-jwxvn" Oct 30 18:38:10.439020 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.438998 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Oct 30 18:38:10.439212 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.439192 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Oct 30 18:38:10.439288 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.439221 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Oct 30 18:38:10.439288 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.439226 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-qqgkz\"" Oct 30 18:38:10.439288 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.439246 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2529f\"" Oct 30 18:38:10.439288 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.439200 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Oct 30 18:38:10.439288 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.439249 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Oct 30 18:38:10.439288 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.439277 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-m9jbf\"" Oct 30 18:38:10.439482 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.439278 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Oct 30 18:38:10.439591 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.439555 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Oct 30 18:38:10.440711 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.440695 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-mrtvb\"" Oct 30 18:38:10.440805 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.440722 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-74dcr\"" Oct 30 18:38:10.440805 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.440732 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Oct 30 18:38:10.448801 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.448786 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fdrkw"] Oct 30 18:38:10.448900 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.448887 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:10.460967 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.460951 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-pvzsh"] Oct 30 18:38:10.461032 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.460978 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-vwws2"] Oct 30 18:38:10.461032 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.460993 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-mm49z"] Oct 30 18:38:10.461032 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.461011 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-td46q"] Oct 30 18:38:10.461032 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.461026 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-6kk4m"] Oct 30 18:38:10.461175 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.461039 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb"] Oct 30 18:38:10.461175 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.461046 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fdrkw" Oct 30 18:38:10.461175 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.461056 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4z4ss"] Oct 30 18:38:10.461175 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.461069 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-797b6dc769-r2cdl"] Oct 30 18:38:10.461175 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.461081 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp"] Oct 30 18:38:10.461175 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.461096 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f6754678-55bz6"] Oct 30 18:38:10.461175 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.461114 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7dbbfc9475-s4vg9"] Oct 30 18:38:10.461175 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.461133 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-gg799"] Oct 30 18:38:10.461175 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.461153 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-hqzbs"] Oct 30 18:38:10.461432 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.461181 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kkzsh"] Oct 30 18:38:10.461432 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.461209 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-jwxvn"] Oct 30 18:38:10.461432 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.461229 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fdrkw"] Oct 30 18:38:10.461432 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.461248 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5"] Oct 30 18:38:10.463057 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.463043 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Oct 30 18:38:10.463420 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.463403 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Oct 30 18:38:10.463491 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.463430 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-kk5cx\"" Oct 30 18:38:10.463491 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.463478 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Oct 30 18:38:10.463491 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.463482 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-t264s\"" Oct 30 18:38:10.463611 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.463538 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-n2msw\"" Oct 30 18:38:10.463639 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.463627 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-hrpgl\"" Oct 30 18:38:10.463754 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.463633 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Oct 30 18:38:10.492714 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.492692 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-trusted-ca\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:10.492811 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.492718 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v87g2\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-kube-api-access-v87g2\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.492811 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.492735 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/872d91f9-1bbc-4db1-bffe-6d6241175fee-installation-pull-secrets\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.492811 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.492754 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-installation-pull-secrets\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.492811 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.492804 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8blzq\" (UniqueName: \"kubernetes.io/projected/89271a7f-e9c4-43dd-a320-54900b3fb8b7-kube-api-access-8blzq\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-689xp\" (UID: \"89271a7f-e9c4-43dd-a320-54900b3fb8b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" Oct 30 18:38:10.492932 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.492853 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:10.492932 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.492884 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:10.492932 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.492902 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4p6x\" (UniqueName: \"kubernetes.io/projected/3180fff2-f289-47e6-a37c-34c5dff50aa2-kube-api-access-n4p6x\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:10.493031 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:10.492930 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:38:10.493031 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.492982 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-service-ca-bundle\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.493031 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493020 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c2tj\" (UniqueName: \"kubernetes.io/projected/9cd97653-a745-4fd6-9a6c-71aaa742b415-kube-api-access-2c2tj\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:10.493127 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:10.493042 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls podName:2797391b-fd3d-4222-82c8-b28a2d16c9e3 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:10.993026075 +0000 UTC m=+32.706627368 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-td46q" (UID: "2797391b-fd3d-4222-82c8-b28a2d16c9e3") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:38:10.493127 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493095 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-bound-sa-token\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.493127 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493123 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-config\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:10.493280 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493153 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5w86\" (UniqueName: \"kubernetes.io/projected/a36f2ae5-c6ca-49cf-a775-82a97e92f2c4-kube-api-access-m5w86\") pod \"kas-connection-checker-685895c7c9-pvzsh\" (UID: \"a36f2ae5-c6ca-49cf-a775-82a97e92f2c4\") " pod="kube-system/kas-connection-checker-685895c7c9-pvzsh" Oct 30 18:38:10.493280 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493169 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-serving-cert\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.493280 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493197 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-serving-cert\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:10.493280 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493234 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-hqzbs\" (UID: \"c135952e-12c3-4a65-bd3d-0d0b1aba49d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" Oct 30 18:38:10.493389 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493340 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e65f9eb5-9e22-4c76-a1b5-e2250003faa6-config\") pod \"service-ca-operator-544d749c8c-xt7x5\" (UID: \"e65f9eb5-9e22-4c76-a1b5-e2250003faa6\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" Oct 30 18:38:10.493389 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493383 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hqzbs\" (UID: \"c135952e-12c3-4a65-bd3d-0d0b1aba49d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" Oct 30 18:38:10.493448 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493407 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:10.493448 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493442 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cst78\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-kube-api-access-cst78\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.493508 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493467 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-image-registry-private-configuration\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.493508 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493478 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-service-ca-bundle\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.493508 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493496 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.493620 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493532 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/89271a7f-e9c4-43dd-a320-54900b3fb8b7-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-689xp\" (UID: \"89271a7f-e9c4-43dd-a320-54900b3fb8b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" Oct 30 18:38:10.493620 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493556 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-stats-auth\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:10.493620 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493604 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2797391b-fd3d-4222-82c8-b28a2d16c9e3-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:10.493717 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493634 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6srj\" (UniqueName: \"kubernetes.io/projected/7a4adea8-c5dc-4a4d-ad54-e985838fa6c5-kube-api-access-w6srj\") pod \"kas-connection-checker-685895c7c9-mm49z\" (UID: \"7a4adea8-c5dc-4a4d-ad54-e985838fa6c5\") " pod="kube-system/kas-connection-checker-685895c7c9-mm49z" Oct 30 18:38:10.493717 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493657 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zfjf\" (UniqueName: \"kubernetes.io/projected/34a1d011-b30b-49b8-bc75-bfb9dbba3817-kube-api-access-4zfjf\") pod \"kas-connection-checker-685895c7c9-vwws2\" (UID: \"34a1d011-b30b-49b8-bc75-bfb9dbba3817\") " pod="kube-system/kas-connection-checker-685895c7c9-vwws2" Oct 30 18:38:10.493717 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493685 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2q9pq\" (UniqueName: \"kubernetes.io/projected/bb52d69e-0e3f-46b5-8491-a2be5a912d29-kube-api-access-2q9pq\") pod \"volume-data-source-validator-58c9f5f46d-4z4ss\" (UID: \"bb52d69e-0e3f-46b5-8491-a2be5a912d29\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4z4ss" Oct 30 18:38:10.493856 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493718 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-snapshots\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.493856 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493792 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a88dde4-2103-476d-81c4-4b255cff4828-ca-trust-extracted\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.493856 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493826 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-default-certificate\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:10.493964 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493854 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/872d91f9-1bbc-4db1-bffe-6d6241175fee-image-registry-private-configuration\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.493964 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493886 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.493964 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493948 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg2js\" (UniqueName: \"kubernetes.io/projected/2797391b-fd3d-4222-82c8-b28a2d16c9e3-kube-api-access-lg2js\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:10.494054 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.493988 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-trusted-ca\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.494113 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.494094 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e65f9eb5-9e22-4c76-a1b5-e2250003faa6-serving-cert\") pod \"service-ca-operator-544d749c8c-xt7x5\" (UID: \"e65f9eb5-9e22-4c76-a1b5-e2250003faa6\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" Oct 30 18:38:10.494160 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.494147 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/872d91f9-1bbc-4db1-bffe-6d6241175fee-trusted-ca\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.494210 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.494191 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-bound-sa-token\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.494261 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.494227 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-snapshots\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.494261 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.494231 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:10.494261 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.494239 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2797391b-fd3d-4222-82c8-b28a2d16c9e3-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:10.494378 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.494302 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:10.494378 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.494353 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-registry-certificates\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.494463 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.494382 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.494463 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.494404 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6srz4\" (UniqueName: \"kubernetes.io/projected/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-kube-api-access-6srz4\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.494567 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.494552 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-certificates\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.494650 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.494631 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/89271a7f-e9c4-43dd-a320-54900b3fb8b7-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-689xp\" (UID: \"89271a7f-e9c4-43dd-a320-54900b3fb8b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" Oct 30 18:38:10.494699 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.494680 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-tmp\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.494749 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.494730 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k44bp\" (UniqueName: \"kubernetes.io/projected/e65f9eb5-9e22-4c76-a1b5-e2250003faa6-kube-api-access-k44bp\") pod \"service-ca-operator-544d749c8c-xt7x5\" (UID: \"e65f9eb5-9e22-4c76-a1b5-e2250003faa6\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" Oct 30 18:38:10.494801 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.494763 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/872d91f9-1bbc-4db1-bffe-6d6241175fee-ca-trust-extracted\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.494801 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.494792 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghbwl\" (UniqueName: \"kubernetes.io/projected/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-kube-api-access-ghbwl\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:10.494985 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.494969 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-tmp\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.495375 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.495353 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.497010 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.496994 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-serving-cert\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.513231 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.513203 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Oct 30 18:38:10.513318 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.513228 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Oct 30 18:38:10.513370 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.513325 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:38:10.513370 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.513356 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Oct 30 18:38:10.513469 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.513395 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 18:38:10.513469 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.513446 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Oct 30 18:38:10.513560 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.513472 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:38:10.513560 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.513472 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 18:38:10.513560 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.513504 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 18:38:10.513560 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.513516 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Oct 30 18:38:10.513560 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.513551 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:38:10.513560 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.513559 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Oct 30 18:38:10.513775 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.513565 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Oct 30 18:38:10.513775 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.513593 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 18:38:10.513775 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.513569 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 18:38:10.513775 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.513726 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Oct 30 18:38:10.515419 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.515401 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Oct 30 18:38:10.515509 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.515423 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Oct 30 18:38:10.539401 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.539378 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 18:38:10.539481 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.539413 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 18:38:10.539481 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.539422 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xhgds\"" Oct 30 18:38:10.539481 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.539478 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-j58tx\"" Oct 30 18:38:10.539600 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.539489 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-dx4x5\"" Oct 30 18:38:10.550033 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.550013 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5w86\" (UniqueName: \"kubernetes.io/projected/a36f2ae5-c6ca-49cf-a775-82a97e92f2c4-kube-api-access-m5w86\") pod \"kas-connection-checker-685895c7c9-pvzsh\" (UID: \"a36f2ae5-c6ca-49cf-a775-82a97e92f2c4\") " pod="kube-system/kas-connection-checker-685895c7c9-pvzsh" Oct 30 18:38:10.550559 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.550506 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6srj\" (UniqueName: \"kubernetes.io/projected/7a4adea8-c5dc-4a4d-ad54-e985838fa6c5-kube-api-access-w6srj\") pod \"kas-connection-checker-685895c7c9-mm49z\" (UID: \"7a4adea8-c5dc-4a4d-ad54-e985838fa6c5\") " pod="kube-system/kas-connection-checker-685895c7c9-mm49z" Oct 30 18:38:10.550869 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.550852 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg2js\" (UniqueName: \"kubernetes.io/projected/2797391b-fd3d-4222-82c8-b28a2d16c9e3-kube-api-access-lg2js\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:10.551314 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.551272 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6srz4\" (UniqueName: \"kubernetes.io/projected/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-kube-api-access-6srz4\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.551543 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.551522 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zfjf\" (UniqueName: \"kubernetes.io/projected/34a1d011-b30b-49b8-bc75-bfb9dbba3817-kube-api-access-4zfjf\") pod \"kas-connection-checker-685895c7c9-vwws2\" (UID: \"34a1d011-b30b-49b8-bc75-bfb9dbba3817\") " pod="kube-system/kas-connection-checker-685895c7c9-vwws2" Oct 30 18:38:10.581286 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.581263 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-pvzsh" Oct 30 18:38:10.591366 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.591350 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-mm49z" Oct 30 18:38:10.595113 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.595091 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-trusted-ca\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.595205 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.595127 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e65f9eb5-9e22-4c76-a1b5-e2250003faa6-serving-cert\") pod \"service-ca-operator-544d749c8c-xt7x5\" (UID: \"e65f9eb5-9e22-4c76-a1b5-e2250003faa6\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" Oct 30 18:38:10.595205 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.595144 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/872d91f9-1bbc-4db1-bffe-6d6241175fee-trusted-ca\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.595205 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.595166 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-bound-sa-token\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.595531 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.595503 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:10.595634 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.595548 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvrqw\" (UniqueName: \"kubernetes.io/projected/51d9053b-abef-45c8-bd1a-67343812dcd9-kube-api-access-lvrqw\") pod \"network-check-source-6f78d59f99-jwxvn\" (UID: \"51d9053b-abef-45c8-bd1a-67343812dcd9\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-jwxvn" Oct 30 18:38:10.595634 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.595591 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-tmp-dir\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:10.595634 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.595612 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:10.595803 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.595635 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-registry-certificates\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.595803 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.595694 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-certificates\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.595803 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:10.595718 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:38:10.595803 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.595727 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63658d25-6d63-42e2-9755-8ca66a322d61-cert\") pod \"ingress-canary-fdrkw\" (UID: \"63658d25-6d63-42e2-9755-8ca66a322d61\") " pod="openshift-ingress-canary/ingress-canary-fdrkw" Oct 30 18:38:10.595803 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:10.595775 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls podName:9cd97653-a745-4fd6-9a6c-71aaa742b415 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:11.095761632 +0000 UTC m=+32.809362955 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-wf7rb" (UID: "9cd97653-a745-4fd6-9a6c-71aaa742b415") : secret "samples-operator-tls" not found Oct 30 18:38:10.596064 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.595837 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/89271a7f-e9c4-43dd-a320-54900b3fb8b7-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-689xp\" (UID: \"89271a7f-e9c4-43dd-a320-54900b3fb8b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" Oct 30 18:38:10.596064 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.595869 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k44bp\" (UniqueName: \"kubernetes.io/projected/e65f9eb5-9e22-4c76-a1b5-e2250003faa6-kube-api-access-k44bp\") pod \"service-ca-operator-544d749c8c-xt7x5\" (UID: \"e65f9eb5-9e22-4c76-a1b5-e2250003faa6\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" Oct 30 18:38:10.596064 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:10.595875 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 18:38:10.596064 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.595896 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/872d91f9-1bbc-4db1-bffe-6d6241175fee-ca-trust-extracted\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.596064 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.595920 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hlr2\" (UniqueName: \"kubernetes.io/projected/63658d25-6d63-42e2-9755-8ca66a322d61-kube-api-access-7hlr2\") pod \"ingress-canary-fdrkw\" (UID: \"63658d25-6d63-42e2-9755-8ca66a322d61\") " pod="openshift-ingress-canary/ingress-canary-fdrkw" Oct 30 18:38:10.596064 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:10.595942 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs podName:3180fff2-f289-47e6-a37c-34c5dff50aa2 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:11.095925901 +0000 UTC m=+32.809527194 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs") pod "router-default-797b6dc769-r2cdl" (UID: "3180fff2-f289-47e6-a37c-34c5dff50aa2") : secret "router-metrics-certs-default" not found Oct 30 18:38:10.596064 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596028 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-trusted-ca\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.596397 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596083 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghbwl\" (UniqueName: \"kubernetes.io/projected/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-kube-api-access-ghbwl\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:10.596397 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596120 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-trusted-ca\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:10.596397 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596146 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v87g2\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-kube-api-access-v87g2\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.596397 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596172 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/872d91f9-1bbc-4db1-bffe-6d6241175fee-installation-pull-secrets\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.596397 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596180 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/872d91f9-1bbc-4db1-bffe-6d6241175fee-trusted-ca\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.596397 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596200 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-installation-pull-secrets\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.596397 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596225 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8blzq\" (UniqueName: \"kubernetes.io/projected/89271a7f-e9c4-43dd-a320-54900b3fb8b7-kube-api-access-8blzq\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-689xp\" (UID: \"89271a7f-e9c4-43dd-a320-54900b3fb8b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" Oct 30 18:38:10.596397 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596285 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v72l5\" (UniqueName: \"kubernetes.io/projected/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-kube-api-access-v72l5\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:10.596397 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596305 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/872d91f9-1bbc-4db1-bffe-6d6241175fee-ca-trust-extracted\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.596397 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596313 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-certificates\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.596397 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596328 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:10.596397 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596354 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4p6x\" (UniqueName: \"kubernetes.io/projected/3180fff2-f289-47e6-a37c-34c5dff50aa2-kube-api-access-n4p6x\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:10.596397 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596396 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c2tj\" (UniqueName: \"kubernetes.io/projected/9cd97653-a745-4fd6-9a6c-71aaa742b415-kube-api-access-2c2tj\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:10.597026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596415 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-registry-certificates\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.597026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596432 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-bound-sa-token\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.597026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596509 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/89271a7f-e9c4-43dd-a320-54900b3fb8b7-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-689xp\" (UID: \"89271a7f-e9c4-43dd-a320-54900b3fb8b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" Oct 30 18:38:10.597026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596656 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-config-volume\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:10.597026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596709 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-config\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:10.597026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596735 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:10.597026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596818 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-serving-cert\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:10.597026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596856 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-hqzbs\" (UID: \"c135952e-12c3-4a65-bd3d-0d0b1aba49d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" Oct 30 18:38:10.597026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596888 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e65f9eb5-9e22-4c76-a1b5-e2250003faa6-config\") pod \"service-ca-operator-544d749c8c-xt7x5\" (UID: \"e65f9eb5-9e22-4c76-a1b5-e2250003faa6\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" Oct 30 18:38:10.597026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596918 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hqzbs\" (UID: \"c135952e-12c3-4a65-bd3d-0d0b1aba49d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" Oct 30 18:38:10.597026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596941 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:10.597026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596960 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-trusted-ca\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:10.597026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.596967 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cst78\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-kube-api-access-cst78\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.597026 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.597014 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-image-registry-private-configuration\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.597698 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.597040 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.597698 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.597064 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/89271a7f-e9c4-43dd-a320-54900b3fb8b7-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-689xp\" (UID: \"89271a7f-e9c4-43dd-a320-54900b3fb8b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" Oct 30 18:38:10.597698 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.597091 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-stats-auth\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:10.597698 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.597125 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q9pq\" (UniqueName: \"kubernetes.io/projected/bb52d69e-0e3f-46b5-8491-a2be5a912d29-kube-api-access-2q9pq\") pod \"volume-data-source-validator-58c9f5f46d-4z4ss\" (UID: \"bb52d69e-0e3f-46b5-8491-a2be5a912d29\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4z4ss" Oct 30 18:38:10.597698 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.597160 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a88dde4-2103-476d-81c4-4b255cff4828-ca-trust-extracted\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.597698 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.597190 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-default-certificate\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:10.597698 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:10.597208 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:38:10.597698 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.597214 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/872d91f9-1bbc-4db1-bffe-6d6241175fee-image-registry-private-configuration\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.597698 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:10.597269 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert podName:c135952e-12c3-4a65-bd3d-0d0b1aba49d1 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:11.097252684 +0000 UTC m=+32.810853979 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-hqzbs" (UID: "c135952e-12c3-4a65-bd3d-0d0b1aba49d1") : secret "networking-console-plugin-cert" not found Oct 30 18:38:10.597698 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.597291 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.597698 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.597294 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-config\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:10.597698 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:10.597420 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:38:10.597698 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:10.597433 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dbbfc9475-s4vg9: secret "image-registry-tls" not found Oct 30 18:38:10.597698 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:10.597471 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls podName:872d91f9-1bbc-4db1-bffe-6d6241175fee nodeName:}" failed. No retries permitted until 2026-10-30 18:38:11.097457244 +0000 UTC m=+32.811058537 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls") pod "image-registry-7dbbfc9475-s4vg9" (UID: "872d91f9-1bbc-4db1-bffe-6d6241175fee") : secret "image-registry-tls" not found Oct 30 18:38:10.597698 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.597542 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e65f9eb5-9e22-4c76-a1b5-e2250003faa6-config\") pod \"service-ca-operator-544d749c8c-xt7x5\" (UID: \"e65f9eb5-9e22-4c76-a1b5-e2250003faa6\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" Oct 30 18:38:10.597698 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.597089 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:10.598472 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:10.598058 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle podName:3180fff2-f289-47e6-a37c-34c5dff50aa2 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:11.098042326 +0000 UTC m=+32.811643639 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle") pod "router-default-797b6dc769-r2cdl" (UID: "3180fff2-f289-47e6-a37c-34c5dff50aa2") : configmap references non-existent config key: service-ca.crt Oct 30 18:38:10.598472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.598089 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-hqzbs\" (UID: \"c135952e-12c3-4a65-bd3d-0d0b1aba49d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" Oct 30 18:38:10.598472 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:10.598168 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:38:10.598472 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:10.598181 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f6754678-55bz6: secret "image-registry-tls" not found Oct 30 18:38:10.598472 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:10.598226 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls podName:4a88dde4-2103-476d-81c4-4b255cff4828 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:11.098213418 +0000 UTC m=+32.811814713 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls") pod "image-registry-7f6754678-55bz6" (UID: "4a88dde4-2103-476d-81c4-4b255cff4828") : secret "image-registry-tls" not found Oct 30 18:38:10.598472 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.598264 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a88dde4-2103-476d-81c4-4b255cff4828-ca-trust-extracted\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.599138 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.599097 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-installation-pull-secrets\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.599977 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.599952 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-serving-cert\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:10.600227 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.600183 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e65f9eb5-9e22-4c76-a1b5-e2250003faa6-serving-cert\") pod \"service-ca-operator-544d749c8c-xt7x5\" (UID: \"e65f9eb5-9e22-4c76-a1b5-e2250003faa6\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" Oct 30 18:38:10.600392 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.600374 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-stats-auth\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:10.600664 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.600637 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/872d91f9-1bbc-4db1-bffe-6d6241175fee-image-registry-private-configuration\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.600713 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.600643 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-image-registry-private-configuration\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.600713 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.600645 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/872d91f9-1bbc-4db1-bffe-6d6241175fee-installation-pull-secrets\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.601257 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.601237 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-default-certificate\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:10.601349 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.601334 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/89271a7f-e9c4-43dd-a320-54900b3fb8b7-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-689xp\" (UID: \"89271a7f-e9c4-43dd-a320-54900b3fb8b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" Oct 30 18:38:10.632416 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.632398 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kas-connection-checker-685895c7c9-vwws2" Oct 30 18:38:10.652360 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.652339 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:10.655390 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.655324 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-bound-sa-token\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.656098 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.655996 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v87g2\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-kube-api-access-v87g2\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.657535 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.657030 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k44bp\" (UniqueName: \"kubernetes.io/projected/e65f9eb5-9e22-4c76-a1b5-e2250003faa6-kube-api-access-k44bp\") pod \"service-ca-operator-544d749c8c-xt7x5\" (UID: \"e65f9eb5-9e22-4c76-a1b5-e2250003faa6\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" Oct 30 18:38:10.657535 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.657488 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c2tj\" (UniqueName: \"kubernetes.io/projected/9cd97653-a745-4fd6-9a6c-71aaa742b415-kube-api-access-2c2tj\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:10.657995 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.657963 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-bound-sa-token\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:10.658524 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.658473 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8blzq\" (UniqueName: \"kubernetes.io/projected/89271a7f-e9c4-43dd-a320-54900b3fb8b7-kube-api-access-8blzq\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-689xp\" (UID: \"89271a7f-e9c4-43dd-a320-54900b3fb8b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" Oct 30 18:38:10.659061 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.659032 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4p6x\" (UniqueName: \"kubernetes.io/projected/3180fff2-f289-47e6-a37c-34c5dff50aa2-kube-api-access-n4p6x\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:10.661511 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.660143 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghbwl\" (UniqueName: \"kubernetes.io/projected/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-kube-api-access-ghbwl\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:10.661511 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.660294 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q9pq\" (UniqueName: \"kubernetes.io/projected/bb52d69e-0e3f-46b5-8491-a2be5a912d29-kube-api-access-2q9pq\") pod \"volume-data-source-validator-58c9f5f46d-4z4ss\" (UID: \"bb52d69e-0e3f-46b5-8491-a2be5a912d29\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4z4ss" Oct 30 18:38:10.661511 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.661289 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cst78\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-kube-api-access-cst78\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:10.692007 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.691976 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4z4ss" Oct 30 18:38:10.699618 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.698846 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvrqw\" (UniqueName: \"kubernetes.io/projected/51d9053b-abef-45c8-bd1a-67343812dcd9-kube-api-access-lvrqw\") pod \"network-check-source-6f78d59f99-jwxvn\" (UID: \"51d9053b-abef-45c8-bd1a-67343812dcd9\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-jwxvn" Oct 30 18:38:10.699618 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.698889 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-tmp-dir\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:10.699618 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.699195 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63658d25-6d63-42e2-9755-8ca66a322d61-cert\") pod \"ingress-canary-fdrkw\" (UID: \"63658d25-6d63-42e2-9755-8ca66a322d61\") " pod="openshift-ingress-canary/ingress-canary-fdrkw" Oct 30 18:38:10.699618 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.699210 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-tmp-dir\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:10.699618 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.699225 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hlr2\" (UniqueName: \"kubernetes.io/projected/63658d25-6d63-42e2-9755-8ca66a322d61-kube-api-access-7hlr2\") pod \"ingress-canary-fdrkw\" (UID: \"63658d25-6d63-42e2-9755-8ca66a322d61\") " pod="openshift-ingress-canary/ingress-canary-fdrkw" Oct 30 18:38:10.699618 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.699270 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v72l5\" (UniqueName: \"kubernetes.io/projected/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-kube-api-access-v72l5\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:10.699618 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.699299 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-config-volume\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:10.699618 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.699322 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:10.699618 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:10.699402 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:38:10.699618 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:10.699439 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls podName:a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f nodeName:}" failed. No retries permitted until 2026-10-30 18:38:11.199426832 +0000 UTC m=+32.913028124 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls") pod "dns-default-kkzsh" (UID: "a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f") : secret "dns-default-metrics-tls" not found Oct 30 18:38:10.700211 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.699969 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-config-volume\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:10.702773 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.702082 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63658d25-6d63-42e2-9755-8ca66a322d61-cert\") pod \"ingress-canary-fdrkw\" (UID: \"63658d25-6d63-42e2-9755-8ca66a322d61\") " pod="openshift-ingress-canary/ingress-canary-fdrkw" Oct 30 18:38:10.710390 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.709390 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" Oct 30 18:38:10.718362 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.717386 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:38:10.736873 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.736847 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:10.741707 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.741684 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:10.758081 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.758053 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" Oct 30 18:38:10.760020 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.759995 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvrqw\" (UniqueName: \"kubernetes.io/projected/51d9053b-abef-45c8-bd1a-67343812dcd9-kube-api-access-lvrqw\") pod \"network-check-source-6f78d59f99-jwxvn\" (UID: \"51d9053b-abef-45c8-bd1a-67343812dcd9\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-jwxvn" Oct 30 18:38:10.761175 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.761150 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v72l5\" (UniqueName: \"kubernetes.io/projected/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-kube-api-access-v72l5\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:10.762625 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.762567 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hlr2\" (UniqueName: \"kubernetes.io/projected/63658d25-6d63-42e2-9755-8ca66a322d61-kube-api-access-7hlr2\") pod \"ingress-canary-fdrkw\" (UID: \"63658d25-6d63-42e2-9755-8ca66a322d61\") " pod="openshift-ingress-canary/ingress-canary-fdrkw" Oct 30 18:38:10.764216 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.764162 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-jwxvn" Oct 30 18:38:10.808790 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.808505 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-9gxzb\"" Oct 30 18:38:10.829172 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:10.828543 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fdrkw" Oct 30 18:38:11.001169 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.001138 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:11.001340 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.001279 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:38:11.001389 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.001377 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls podName:2797391b-fd3d-4222-82c8-b28a2d16c9e3 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:12.001362603 +0000 UTC m=+33.714963899 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-td46q" (UID: "2797391b-fd3d-4222-82c8-b28a2d16c9e3") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:38:11.032015 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.031987 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-vwws2"] Oct 30 18:38:11.034842 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.034819 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-pvzsh"] Oct 30 18:38:11.039758 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.039735 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/kas-connection-checker-685895c7c9-mm49z"] Oct 30 18:38:11.042401 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:38:11.042375 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a4adea8_c5dc_4a4d_ad54_e985838fa6c5.slice/crio-28a4b9b1d84bfb15c36875674bcc12c565c22f842974116ffe42f7d4f3c326a0 WatchSource:0}: Error finding container 28a4b9b1d84bfb15c36875674bcc12c565c22f842974116ffe42f7d4f3c326a0: Status 404 returned error can't find the container with id 28a4b9b1d84bfb15c36875674bcc12c565c22f842974116ffe42f7d4f3c326a0 Oct 30 18:38:11.071425 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.071396 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-7c4986d449-6kk4m"] Oct 30 18:38:11.080184 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:38:11.080154 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ca07043_d7b5_4660_87d4_2a92bf5dacbf.slice/crio-9809304bf3e9a708db1a5b926a67a728cc1e9c0f3308b9f01a9f57dd56787f89 WatchSource:0}: Error finding container 9809304bf3e9a708db1a5b926a67a728cc1e9c0f3308b9f01a9f57dd56787f89: Status 404 returned error can't find the container with id 9809304bf3e9a708db1a5b926a67a728cc1e9c0f3308b9f01a9f57dd56787f89 Oct 30 18:38:11.102767 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.102056 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:11.102767 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.102108 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:11.102767 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.102149 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:11.102767 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.102294 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hqzbs\" (UID: \"c135952e-12c3-4a65-bd3d-0d0b1aba49d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" Oct 30 18:38:11.102767 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.102321 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:11.102767 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.102361 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:11.102767 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.102470 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:38:11.102767 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.102492 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:38:11.102767 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.102506 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f6754678-55bz6: secret "image-registry-tls" not found Oct 30 18:38:11.102767 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.102534 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls podName:9cd97653-a745-4fd6-9a6c-71aaa742b415 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:12.102514829 +0000 UTC m=+33.816116135 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-wf7rb" (UID: "9cd97653-a745-4fd6-9a6c-71aaa742b415") : secret "samples-operator-tls" not found Oct 30 18:38:11.102767 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.102554 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls podName:4a88dde4-2103-476d-81c4-4b255cff4828 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:12.102543602 +0000 UTC m=+33.816144897 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls") pod "image-registry-7f6754678-55bz6" (UID: "4a88dde4-2103-476d-81c4-4b255cff4828") : secret "image-registry-tls" not found Oct 30 18:38:11.102767 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.102607 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:38:11.102767 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.102641 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:38:11.102767 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.102653 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dbbfc9475-s4vg9: secret "image-registry-tls" not found Oct 30 18:38:11.102767 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.102652 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert podName:c135952e-12c3-4a65-bd3d-0d0b1aba49d1 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:12.102634673 +0000 UTC m=+33.816235968 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-hqzbs" (UID: "c135952e-12c3-4a65-bd3d-0d0b1aba49d1") : secret "networking-console-plugin-cert" not found Oct 30 18:38:11.102767 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.102695 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls podName:872d91f9-1bbc-4db1-bffe-6d6241175fee nodeName:}" failed. No retries permitted until 2026-10-30 18:38:12.102684316 +0000 UTC m=+33.816285608 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls") pod "image-registry-7dbbfc9475-s4vg9" (UID: "872d91f9-1bbc-4db1-bffe-6d6241175fee") : secret "image-registry-tls" not found Oct 30 18:38:11.103593 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.102720 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle podName:3180fff2-f289-47e6-a37c-34c5dff50aa2 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:12.102709635 +0000 UTC m=+33.816310945 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle") pod "router-default-797b6dc769-r2cdl" (UID: "3180fff2-f289-47e6-a37c-34c5dff50aa2") : configmap references non-existent config key: service-ca.crt Oct 30 18:38:11.103593 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.102792 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 18:38:11.103593 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.102832 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs podName:3180fff2-f289-47e6-a37c-34c5dff50aa2 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:12.102821323 +0000 UTC m=+33.816422638 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs") pod "router-default-797b6dc769-r2cdl" (UID: "3180fff2-f289-47e6-a37c-34c5dff50aa2") : secret "router-metrics-certs-default" not found Oct 30 18:38:11.133146 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.133119 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4z4ss"] Oct 30 18:38:11.135426 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.135399 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5"] Oct 30 18:38:11.149003 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:38:11.148977 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb52d69e_0e3f_46b5_8491_a2be5a912d29.slice/crio-6308f380c32bcbf4b59577d164b1513e052edcfddce9b3a9e4e0392784d389ae WatchSource:0}: Error finding container 6308f380c32bcbf4b59577d164b1513e052edcfddce9b3a9e4e0392784d389ae: Status 404 returned error can't find the container with id 6308f380c32bcbf4b59577d164b1513e052edcfddce9b3a9e4e0392784d389ae Oct 30 18:38:11.149202 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:38:11.149184 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode65f9eb5_9e22_4c76_a1b5_e2250003faa6.slice/crio-e09c65370ab654a610997634985c1f985d1ca8b2f217adcfd54b7bbd0af1af48 WatchSource:0}: Error finding container e09c65370ab654a610997634985c1f985d1ca8b2f217adcfd54b7bbd0af1af48: Status 404 returned error can't find the container with id e09c65370ab654a610997634985c1f985d1ca8b2f217adcfd54b7bbd0af1af48 Oct 30 18:38:11.185331 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.185303 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp"] Oct 30 18:38:11.190317 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.190262 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-6f78d59f99-jwxvn"] Oct 30 18:38:11.191085 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.191066 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7cd56b8dbc-gg799"] Oct 30 18:38:11.205160 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.204925 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:11.205160 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.205063 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:38:11.205160 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.205119 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls podName:a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f nodeName:}" failed. No retries permitted until 2026-10-30 18:38:12.205098988 +0000 UTC m=+33.918700294 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls") pod "dns-default-kkzsh" (UID: "a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f") : secret "dns-default-metrics-tls" not found Oct 30 18:38:11.244060 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.244032 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fdrkw"] Oct 30 18:38:11.247583 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:38:11.247541 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63658d25_6d63_42e2_9755_8ca66a322d61.slice/crio-10e2ded499f32c29040bcaac33da17b080ba2bf5682452b99e2e12d16855c1fe WatchSource:0}: Error finding container 10e2ded499f32c29040bcaac33da17b080ba2bf5682452b99e2e12d16855c1fe: Status 404 returned error can't find the container with id 10e2ded499f32c29040bcaac33da17b080ba2bf5682452b99e2e12d16855c1fe Oct 30 18:38:11.710640 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.710603 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:38:11.710807 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.710652 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs\") pod \"network-metrics-daemon-2s5rm\" (UID: \"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2\") " pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:38:11.710807 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.710764 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Oct 30 18:38:11.710884 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.710833 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret podName:8c9b04e2-b099-4704-8786-b64a8991d0f4 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:43.710819974 +0000 UTC m=+65.424421283 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret") pod "global-pull-secret-syncer-hmzzs" (UID: "8c9b04e2-b099-4704-8786-b64a8991d0f4") : object "kube-system"/"original-pull-secret" not registered Oct 30 18:38:11.710884 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.710770 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:38:11.710884 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:11.710871 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs podName:8d3a00e6-202e-4ffb-b8e0-67f15d831ae2 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:43.710863041 +0000 UTC m=+65.424464332 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs") pod "network-metrics-daemon-2s5rm" (UID: "8d3a00e6-202e-4ffb-b8e0-67f15d831ae2") : object "openshift-multus"/"metrics-daemon-secret" not registered Oct 30 18:38:11.714643 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.714627 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:38:11.714793 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.714766 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:38:11.766841 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.766808 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 18:38:11.766949 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.766896 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6nrlk\"" Oct 30 18:38:11.767006 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.766948 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-cts7j\"" Oct 30 18:38:11.767006 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.766992 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 18:38:11.812007 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.811981 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l7k8\" (UniqueName: \"kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8\") pod \"network-check-target-95g7n\" (UID: \"cd195fdf-8022-4279-8eca-15d145d59038\") " pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:38:11.814495 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.814471 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l7k8\" (UniqueName: \"kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8\") pod \"network-check-target-95g7n\" (UID: \"cd195fdf-8022-4279-8eca-15d145d59038\") " pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:38:11.927722 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.927684 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-mm49z" event={"ID":"7a4adea8-c5dc-4a4d-ad54-e985838fa6c5","Type":"ContainerStarted","Data":"8967a0f87ed997a8665287a50c01c83fb2b800465871f2fb43a206c2cd1b7516"} Oct 30 18:38:11.927896 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.927772 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-mm49z" event={"ID":"7a4adea8-c5dc-4a4d-ad54-e985838fa6c5","Type":"ContainerStarted","Data":"28a4b9b1d84bfb15c36875674bcc12c565c22f842974116ffe42f7d4f3c326a0"} Oct 30 18:38:11.929242 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.929216 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" event={"ID":"67b57c01-e4a3-4ae1-ab6e-9b386e181adb","Type":"ContainerStarted","Data":"e9aa7c0d77120340e0ef3c531e83d699e403f7825a9c67d1ee67556aa802cc05"} Oct 30 18:38:11.930327 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.930304 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-jwxvn" event={"ID":"51d9053b-abef-45c8-bd1a-67343812dcd9","Type":"ContainerStarted","Data":"b6b8b467a9a9c17d405b5f134e0a4f52618908c014498e12875220ea4be2ae9e"} Oct 30 18:38:11.931429 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.931407 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" event={"ID":"e65f9eb5-9e22-4c76-a1b5-e2250003faa6","Type":"ContainerStarted","Data":"e09c65370ab654a610997634985c1f985d1ca8b2f217adcfd54b7bbd0af1af48"} Oct 30 18:38:11.932811 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.932789 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-pvzsh" event={"ID":"a36f2ae5-c6ca-49cf-a775-82a97e92f2c4","Type":"ContainerStarted","Data":"98e2eb9d7f7e0c261565c88e4c468d06b2f043469faeca06211e4921cc6fc642"} Oct 30 18:38:11.932935 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.932825 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-pvzsh" event={"ID":"a36f2ae5-c6ca-49cf-a775-82a97e92f2c4","Type":"ContainerStarted","Data":"6a71eac430b3a11c70f0218e4d5783578dcf216b23d800a92b73ec44e02196c5"} Oct 30 18:38:11.933888 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.933870 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fdrkw" event={"ID":"63658d25-6d63-42e2-9755-8ca66a322d61","Type":"ContainerStarted","Data":"10e2ded499f32c29040bcaac33da17b080ba2bf5682452b99e2e12d16855c1fe"} Oct 30 18:38:11.934884 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.934865 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4z4ss" event={"ID":"bb52d69e-0e3f-46b5-8491-a2be5a912d29","Type":"ContainerStarted","Data":"6308f380c32bcbf4b59577d164b1513e052edcfddce9b3a9e4e0392784d389ae"} Oct 30 18:38:11.936115 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.936096 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-vwws2" event={"ID":"34a1d011-b30b-49b8-bc75-bfb9dbba3817","Type":"ContainerStarted","Data":"d3f64c7b25b82c32f340dad5270b948b02435e834bee48da6908a21189d0ddcc"} Oct 30 18:38:11.936260 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.936126 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kas-connection-checker-685895c7c9-vwws2" event={"ID":"34a1d011-b30b-49b8-bc75-bfb9dbba3817","Type":"ContainerStarted","Data":"bdda3f2b9e852331ec036d72d43573711d031903faddb053be331836e4cf3d6d"} Oct 30 18:38:11.937112 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.937094 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" event={"ID":"89271a7f-e9c4-43dd-a320-54900b3fb8b7","Type":"ContainerStarted","Data":"756fad8be1817e3079ad90464555d4d70a5101b1568c397ddb5ec583a41bbe42"} Oct 30 18:38:11.938014 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.937995 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-6kk4m" event={"ID":"7ca07043-d7b5-4660-87d4-2a92bf5dacbf","Type":"ContainerStarted","Data":"9809304bf3e9a708db1a5b926a67a728cc1e9c0f3308b9f01a9f57dd56787f89"} Oct 30 18:38:11.959444 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:11.959423 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:38:12.015987 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:12.014013 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:12.015987 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:12.014738 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:38:12.015987 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:12.014816 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls podName:2797391b-fd3d-4222-82c8-b28a2d16c9e3 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:14.014795003 +0000 UTC m=+35.728396319 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-td46q" (UID: "2797391b-fd3d-4222-82c8-b28a2d16c9e3") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:38:12.115331 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:12.115291 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:12.115896 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:12.115350 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:12.115896 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:12.115386 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:12.115896 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:12.115466 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:38:12.115896 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:12.115489 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dbbfc9475-s4vg9: secret "image-registry-tls" not found Oct 30 18:38:12.115896 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:12.115520 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 18:38:12.115896 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:12.115552 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls podName:872d91f9-1bbc-4db1-bffe-6d6241175fee nodeName:}" failed. No retries permitted until 2026-10-30 18:38:14.115533367 +0000 UTC m=+35.829134661 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls") pod "image-registry-7dbbfc9475-s4vg9" (UID: "872d91f9-1bbc-4db1-bffe-6d6241175fee") : secret "image-registry-tls" not found Oct 30 18:38:12.115896 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:12.115623 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs podName:3180fff2-f289-47e6-a37c-34c5dff50aa2 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:14.1155689 +0000 UTC m=+35.829170215 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs") pod "router-default-797b6dc769-r2cdl" (UID: "3180fff2-f289-47e6-a37c-34c5dff50aa2") : secret "router-metrics-certs-default" not found Oct 30 18:38:12.115896 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:12.115626 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:38:12.115896 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:12.115654 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:38:12.115896 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:12.115525 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hqzbs\" (UID: \"c135952e-12c3-4a65-bd3d-0d0b1aba49d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" Oct 30 18:38:12.115896 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:12.115674 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert podName:c135952e-12c3-4a65-bd3d-0d0b1aba49d1 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:14.115659544 +0000 UTC m=+35.829260839 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-hqzbs" (UID: "c135952e-12c3-4a65-bd3d-0d0b1aba49d1") : secret "networking-console-plugin-cert" not found Oct 30 18:38:12.115896 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:12.115758 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls podName:9cd97653-a745-4fd6-9a6c-71aaa742b415 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:14.115746088 +0000 UTC m=+35.829347393 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-wf7rb" (UID: "9cd97653-a745-4fd6-9a6c-71aaa742b415") : secret "samples-operator-tls" not found Oct 30 18:38:12.115896 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:12.115780 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:12.115896 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:12.115821 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:12.116699 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:12.115935 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:38:12.116699 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:12.115947 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f6754678-55bz6: secret "image-registry-tls" not found Oct 30 18:38:12.116699 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:12.115947 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle podName:3180fff2-f289-47e6-a37c-34c5dff50aa2 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:14.115934351 +0000 UTC m=+35.829535657 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle") pod "router-default-797b6dc769-r2cdl" (UID: "3180fff2-f289-47e6-a37c-34c5dff50aa2") : configmap references non-existent config key: service-ca.crt Oct 30 18:38:12.116699 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:12.115994 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls podName:4a88dde4-2103-476d-81c4-4b255cff4828 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:14.115981296 +0000 UTC m=+35.829582595 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls") pod "image-registry-7f6754678-55bz6" (UID: "4a88dde4-2103-476d-81c4-4b255cff4828") : secret "image-registry-tls" not found Oct 30 18:38:12.217087 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:12.217048 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:12.217326 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:12.217214 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:38:12.217326 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:12.217289 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls podName:a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f nodeName:}" failed. No retries permitted until 2026-10-30 18:38:14.217267291 +0000 UTC m=+35.930868584 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls") pod "dns-default-kkzsh" (UID: "a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f") : secret "dns-default-metrics-tls" not found Oct 30 18:38:12.333045 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:12.331643 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-pvzsh" podStartSLOduration=266.331623133 podStartE2EDuration="4m26.331623133s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:33:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:12.31342555 +0000 UTC m=+34.027026867" watchObservedRunningTime="2026-10-30 18:38:12.331623133 +0000 UTC m=+34.045224449" Oct 30 18:38:12.333045 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:12.331873 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-95g7n"] Oct 30 18:38:12.338097 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:38:12.338064 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd195fdf_8022_4279_8eca_15d145d59038.slice/crio-53177fd6712175b6d60e8410ffa77d7e8f380168e452c864291dfbcb52a18432 WatchSource:0}: Error finding container 53177fd6712175b6d60e8410ffa77d7e8f380168e452c864291dfbcb52a18432: Status 404 returned error can't find the container with id 53177fd6712175b6d60e8410ffa77d7e8f380168e452c864291dfbcb52a18432 Oct 30 18:38:12.426688 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:12.426633 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-vwws2" podStartSLOduration=266.426614468 podStartE2EDuration="4m26.426614468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:33:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:12.425971127 +0000 UTC m=+34.139572442" watchObservedRunningTime="2026-10-30 18:38:12.426614468 +0000 UTC m=+34.140215782" Oct 30 18:38:12.538438 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:12.538378 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kas-connection-checker-685895c7c9-mm49z" podStartSLOduration=266.538359914 podStartE2EDuration="4m26.538359914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:33:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:12.537767885 +0000 UTC m=+34.251369200" watchObservedRunningTime="2026-10-30 18:38:12.538359914 +0000 UTC m=+34.251961228" Oct 30 18:38:12.946911 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:12.946640 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-95g7n" event={"ID":"cd195fdf-8022-4279-8eca-15d145d59038","Type":"ContainerStarted","Data":"53177fd6712175b6d60e8410ffa77d7e8f380168e452c864291dfbcb52a18432"} Oct 30 18:38:13.953493 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:13.953453 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4z4ss" event={"ID":"bb52d69e-0e3f-46b5-8491-a2be5a912d29","Type":"ContainerStarted","Data":"2d4d0b8938416033bc72d0719e1c778b939944b8d2bcaa6d68072e7ba66239bc"} Oct 30 18:38:14.038022 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:14.037887 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:14.038231 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:14.038043 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:38:14.038231 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:14.038114 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls podName:2797391b-fd3d-4222-82c8-b28a2d16c9e3 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:18.038094891 +0000 UTC m=+39.751696183 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-td46q" (UID: "2797391b-fd3d-4222-82c8-b28a2d16c9e3") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:38:14.069129 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:14.069068 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4z4ss" podStartSLOduration=236.862903999 podStartE2EDuration="3m59.069048317s" totalImagesPullingTime="2.206144318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:14.067359985 +0000 UTC m=+35.780961301" watchObservedRunningTime="2026-10-30 18:38:14.069048317 +0000 UTC m=+35.782649633" Oct 30 18:38:14.139416 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:14.139378 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hqzbs\" (UID: \"c135952e-12c3-4a65-bd3d-0d0b1aba49d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" Oct 30 18:38:14.139612 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:14.139429 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:14.139612 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:14.139459 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:14.139612 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:14.139516 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:14.139612 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:14.139551 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:14.139612 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:14.139598 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:14.139863 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:14.139735 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:38:14.139863 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:14.139793 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls podName:9cd97653-a745-4fd6-9a6c-71aaa742b415 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:18.139775567 +0000 UTC m=+39.853376869 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-wf7rb" (UID: "9cd97653-a745-4fd6-9a6c-71aaa742b415") : secret "samples-operator-tls" not found Oct 30 18:38:14.140230 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:14.140212 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:38:14.140315 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:14.140257 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert podName:c135952e-12c3-4a65-bd3d-0d0b1aba49d1 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:18.140246686 +0000 UTC m=+39.853847978 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-hqzbs" (UID: "c135952e-12c3-4a65-bd3d-0d0b1aba49d1") : secret "networking-console-plugin-cert" not found Oct 30 18:38:14.140315 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:14.140305 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle podName:3180fff2-f289-47e6-a37c-34c5dff50aa2 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:18.14029761 +0000 UTC m=+39.853898902 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle") pod "router-default-797b6dc769-r2cdl" (UID: "3180fff2-f289-47e6-a37c-34c5dff50aa2") : configmap references non-existent config key: service-ca.crt Oct 30 18:38:14.140399 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:14.140344 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:38:14.140399 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:14.140351 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f6754678-55bz6: secret "image-registry-tls" not found Oct 30 18:38:14.140399 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:14.140369 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls podName:4a88dde4-2103-476d-81c4-4b255cff4828 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:18.140363215 +0000 UTC m=+39.853964507 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls") pod "image-registry-7f6754678-55bz6" (UID: "4a88dde4-2103-476d-81c4-4b255cff4828") : secret "image-registry-tls" not found Oct 30 18:38:14.140935 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:14.140918 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:38:14.140935 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:14.140935 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dbbfc9475-s4vg9: secret "image-registry-tls" not found Oct 30 18:38:14.141077 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:14.140966 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls podName:872d91f9-1bbc-4db1-bffe-6d6241175fee nodeName:}" failed. No retries permitted until 2026-10-30 18:38:18.140953369 +0000 UTC m=+39.854554662 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls") pod "image-registry-7dbbfc9475-s4vg9" (UID: "872d91f9-1bbc-4db1-bffe-6d6241175fee") : secret "image-registry-tls" not found Oct 30 18:38:14.141077 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:14.141009 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 18:38:14.141077 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:14.141030 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs podName:3180fff2-f289-47e6-a37c-34c5dff50aa2 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:18.14102314 +0000 UTC m=+39.854624432 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs") pod "router-default-797b6dc769-r2cdl" (UID: "3180fff2-f289-47e6-a37c-34c5dff50aa2") : secret "router-metrics-certs-default" not found Oct 30 18:38:14.206874 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:14.206777 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:14.240615 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:14.240197 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:14.240615 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:14.240502 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:38:14.240615 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:14.240563 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls podName:a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f nodeName:}" failed. No retries permitted until 2026-10-30 18:38:18.240546106 +0000 UTC m=+39.954147405 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls") pod "dns-default-kkzsh" (UID: "a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f") : secret "dns-default-metrics-tls" not found Oct 30 18:38:14.380143 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:14.380070 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:16.969611 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:16.968097 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" event={"ID":"67b57c01-e4a3-4ae1-ab6e-9b386e181adb","Type":"ContainerStarted","Data":"516be20b73ce53a6212a7e9c428334f320249cb96f8b1990d01a624dca65e269"} Oct 30 18:38:16.969611 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:16.969045 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:16.970086 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:16.969785 2578 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-gg799 container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.17:8443/readyz\": dial tcp 10.132.0.17:8443: connect: connection refused" start-of-body= Oct 30 18:38:16.970086 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:16.969833 2578 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" podUID="67b57c01-e4a3-4ae1-ab6e-9b386e181adb" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.17:8443/readyz\": dial tcp 10.132.0.17:8443: connect: connection refused" Oct 30 18:38:16.972063 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:16.971993 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" event={"ID":"e65f9eb5-9e22-4c76-a1b5-e2250003faa6","Type":"ContainerStarted","Data":"b8f824433003d6bed1a3f3b7317e82b3ca4dad9a02be9fbe9e6e1e510d9be232"} Oct 30 18:38:16.974198 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:16.973980 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" event={"ID":"89271a7f-e9c4-43dd-a320-54900b3fb8b7","Type":"ContainerStarted","Data":"cbf5a74d4fe9a24e94f0a15f52455e21ede7e50440a9df950ebd217e34bb0b7e"} Oct 30 18:38:16.976784 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:16.976340 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-6kk4m" event={"ID":"7ca07043-d7b5-4660-87d4-2a92bf5dacbf","Type":"ContainerStarted","Data":"144cbb89729ed1c73163dfde2cbe9d352e2d6f35c4a8dd0fd0fe6a7e0cafd3bf"} Oct 30 18:38:17.198748 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:17.197471 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" podStartSLOduration=236.653264253 podStartE2EDuration="4m2.197450805s" totalImagesPullingTime="5.544186552s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:17.084189822 +0000 UTC m=+38.797791136" watchObservedRunningTime="2026-10-30 18:38:17.197450805 +0000 UTC m=+38.911052121" Oct 30 18:38:17.198748 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:17.197827 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" podStartSLOduration=236.570118863 podStartE2EDuration="4m2.197818955s" totalImagesPullingTime="5.627700092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:17.197053383 +0000 UTC m=+38.910654698" watchObservedRunningTime="2026-10-30 18:38:17.197818955 +0000 UTC m=+38.911420274" Oct 30 18:38:17.311919 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:17.311864 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" podStartSLOduration=236.771104476 podStartE2EDuration="4m2.31184927s" totalImagesPullingTime="5.540744794s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:17.3105301 +0000 UTC m=+39.024131414" watchObservedRunningTime="2026-10-30 18:38:17.31184927 +0000 UTC m=+39.025450620" Oct 30 18:38:17.426703 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:17.426630 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-operator-7c4986d449-6kk4m" podStartSLOduration=236.749627701 podStartE2EDuration="4m2.42661189s" totalImagesPullingTime="5.676984189s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:17.425332295 +0000 UTC m=+39.138933610" watchObservedRunningTime="2026-10-30 18:38:17.42661189 +0000 UTC m=+39.140213234" Oct 30 18:38:17.984358 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:17.984329 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fdrkw" event={"ID":"63658d25-6d63-42e2-9755-8ca66a322d61","Type":"ContainerStarted","Data":"db903bb5be84db7610ed19ac69249b8ec999b999324cdd9c7584b72d00b9f49b"} Oct 30 18:38:17.986125 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:17.986107 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/0.log" Oct 30 18:38:17.986205 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:17.986143 2578 generic.go:356] "Generic (PLEG): container finished" podID="67b57c01-e4a3-4ae1-ab6e-9b386e181adb" containerID="516be20b73ce53a6212a7e9c428334f320249cb96f8b1990d01a624dca65e269" exitCode=255 Oct 30 18:38:17.986253 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:17.986208 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" event={"ID":"67b57c01-e4a3-4ae1-ab6e-9b386e181adb","Type":"ContainerDied","Data":"516be20b73ce53a6212a7e9c428334f320249cb96f8b1990d01a624dca65e269"} Oct 30 18:38:17.986438 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:17.986421 2578 scope.go:122] "RemoveContainer" containerID="516be20b73ce53a6212a7e9c428334f320249cb96f8b1990d01a624dca65e269" Oct 30 18:38:18.082359 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.080783 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:18.082359 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.080884 2578 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Oct 30 18:38:18.082359 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.080945 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls podName:2797391b-fd3d-4222-82c8-b28a2d16c9e3 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:26.080925749 +0000 UTC m=+47.794527056 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-884576c57-td46q" (UID: "2797391b-fd3d-4222-82c8-b28a2d16c9e3") : secret "cluster-monitoring-operator-tls" not found Oct 30 18:38:18.098122 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.098041 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fdrkw" podStartSLOduration=2.200292488 podStartE2EDuration="8.098024226s" totalImagesPullingTime="5.897731738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:18.096251978 +0000 UTC m=+39.809853304" watchObservedRunningTime="2026-10-30 18:38:18.098024226 +0000 UTC m=+39.811625545" Oct 30 18:38:18.182043 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.182017 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:18.182183 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.182055 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:18.182183 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.182076 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:18.182319 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.182178 2578 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Oct 30 18:38:18.182319 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.182197 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hqzbs\" (UID: \"c135952e-12c3-4a65-bd3d-0d0b1aba49d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" Oct 30 18:38:18.182319 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.182225 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:18.182319 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.182242 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs podName:3180fff2-f289-47e6-a37c-34c5dff50aa2 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:26.182224876 +0000 UTC m=+47.895826187 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs") pod "router-default-797b6dc769-r2cdl" (UID: "3180fff2-f289-47e6-a37c-34c5dff50aa2") : secret "router-metrics-certs-default" not found Oct 30 18:38:18.182319 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.182254 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:38:18.182319 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.182271 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:18.182319 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.182319 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls podName:9cd97653-a745-4fd6-9a6c-71aaa742b415 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:26.182301484 +0000 UTC m=+47.895902789 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-wf7rb" (UID: "9cd97653-a745-4fd6-9a6c-71aaa742b415") : secret "samples-operator-tls" not found Oct 30 18:38:18.182589 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.182328 2578 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Oct 30 18:38:18.182589 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.182355 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:38:18.182589 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.182365 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f6754678-55bz6: secret "image-registry-tls" not found Oct 30 18:38:18.182589 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.182179 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:38:18.182589 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.182400 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dbbfc9475-s4vg9: secret "image-registry-tls" not found Oct 30 18:38:18.182589 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.182385 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert podName:c135952e-12c3-4a65-bd3d-0d0b1aba49d1 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:26.18237143 +0000 UTC m=+47.895972730 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert") pod "networking-console-plugin-86c478bb88-hqzbs" (UID: "c135952e-12c3-4a65-bd3d-0d0b1aba49d1") : secret "networking-console-plugin-cert" not found Oct 30 18:38:18.182589 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.182433 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle podName:3180fff2-f289-47e6-a37c-34c5dff50aa2 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:26.182424899 +0000 UTC m=+47.896026194 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle") pod "router-default-797b6dc769-r2cdl" (UID: "3180fff2-f289-47e6-a37c-34c5dff50aa2") : configmap references non-existent config key: service-ca.crt Oct 30 18:38:18.182589 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.182445 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls podName:4a88dde4-2103-476d-81c4-4b255cff4828 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:26.182439363 +0000 UTC m=+47.896040655 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls") pod "image-registry-7f6754678-55bz6" (UID: "4a88dde4-2103-476d-81c4-4b255cff4828") : secret "image-registry-tls" not found Oct 30 18:38:18.182589 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.182456 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls podName:872d91f9-1bbc-4db1-bffe-6d6241175fee nodeName:}" failed. No retries permitted until 2026-10-30 18:38:26.182449682 +0000 UTC m=+47.896050974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls") pod "image-registry-7dbbfc9475-s4vg9" (UID: "872d91f9-1bbc-4db1-bffe-6d6241175fee") : secret "image-registry-tls" not found Oct 30 18:38:18.227942 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.227915 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5bn4v"] Oct 30 18:38:18.249446 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.249394 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:18.282968 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.282938 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d5e33b3-69cd-4fc7-8366-14ffa038713e-data-volume\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:18.283061 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.283001 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:18.283128 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.283109 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4w8p5\" (UniqueName: \"kubernetes.io/projected/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-api-access-4w8p5\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:18.283180 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.283168 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:18.283222 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.283200 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:18.283256 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.283229 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:38:18.283288 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.283279 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls podName:a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f nodeName:}" failed. No retries permitted until 2026-10-30 18:38:26.283265718 +0000 UTC m=+47.996867011 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls") pod "dns-default-kkzsh" (UID: "a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f") : secret "dns-default-metrics-tls" not found Oct 30 18:38:18.283340 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.283324 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3d5e33b3-69cd-4fc7-8366-14ffa038713e-crio-socket\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:18.302169 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.302148 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 18:38:18.302169 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.302148 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 18:38:18.302312 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.302205 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bhm78\"" Oct 30 18:38:18.324372 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.324349 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5bn4v"] Oct 30 18:38:18.383653 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.383629 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3d5e33b3-69cd-4fc7-8366-14ffa038713e-crio-socket\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:18.383741 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.383665 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d5e33b3-69cd-4fc7-8366-14ffa038713e-data-volume\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:18.383779 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.383739 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3d5e33b3-69cd-4fc7-8366-14ffa038713e-crio-socket\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:18.383779 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.383739 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w8p5\" (UniqueName: \"kubernetes.io/projected/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-api-access-4w8p5\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:18.383839 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.383824 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:18.383868 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.383858 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:18.383955 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.383943 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 18:38:18.383997 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.383949 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:38:18.383997 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.383996 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-rbac-proxy-cm podName:3d5e33b3-69cd-4fc7-8366-14ffa038713e nodeName:}" failed. No retries permitted until 2026-10-30 18:38:18.883976754 +0000 UTC m=+40.597578046 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5bn4v" (UID: "3d5e33b3-69cd-4fc7-8366-14ffa038713e") : configmap "kube-rbac-proxy" not found Oct 30 18:38:18.384066 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.384008 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls podName:3d5e33b3-69cd-4fc7-8366-14ffa038713e nodeName:}" failed. No retries permitted until 2026-10-30 18:38:18.884002516 +0000 UTC m=+40.597603807 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5bn4v" (UID: "3d5e33b3-69cd-4fc7-8366-14ffa038713e") : secret "insights-runtime-extractor-tls" not found Oct 30 18:38:18.384066 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.384028 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d5e33b3-69cd-4fc7-8366-14ffa038713e-data-volume\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:18.441050 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.441027 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4w8p5\" (UniqueName: \"kubernetes.io/projected/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-api-access-4w8p5\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:18.483174 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.483146 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-jmjn6-ghcr6"] Oct 30 18:38:18.515664 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.515607 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:18.517106 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.517081 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:18.545650 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.545625 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-jmjn6-ghcr6"] Oct 30 18:38:18.584890 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.584858 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/290aeefa-831a-4e2d-bc4c-11defdfee2d0-archives-path\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:18.585057 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.584931 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwgvn\" (UniqueName: \"kubernetes.io/projected/290aeefa-831a-4e2d-bc4c-11defdfee2d0-kube-api-access-qwgvn\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:18.585057 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.584987 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-trusted-ca-bundle\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:18.585057 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.585046 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-service-ca-bundle\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:18.686309 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.686272 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/290aeefa-831a-4e2d-bc4c-11defdfee2d0-archives-path\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:18.686470 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.686429 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwgvn\" (UniqueName: \"kubernetes.io/projected/290aeefa-831a-4e2d-bc4c-11defdfee2d0-kube-api-access-qwgvn\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:18.686470 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.686458 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-trusted-ca-bundle\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:18.686604 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.686521 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-service-ca-bundle\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:18.686741 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.686718 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/290aeefa-831a-4e2d-bc4c-11defdfee2d0-archives-path\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:18.687094 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.687075 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-service-ca-bundle\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:18.687551 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.687530 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-trusted-ca-bundle\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:18.709784 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.709756 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:18.727698 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.727674 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-j75jm"] Oct 30 18:38:18.749004 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.748973 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwgvn\" (UniqueName: \"kubernetes.io/projected/290aeefa-831a-4e2d-bc4c-11defdfee2d0-kube-api-access-qwgvn\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:18.761153 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.761133 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-fjdk5"] Oct 30 18:38:18.761254 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.761184 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j75jm" Oct 30 18:38:18.781275 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.781250 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fjdk5" Oct 30 18:38:18.787254 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.787229 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh5hg\" (UniqueName: \"kubernetes.io/projected/74990028-c4d9-40d7-b1dc-b45b9f3ccbf2-kube-api-access-xh5hg\") pod \"migrator-66df5d496b-j75jm\" (UID: \"74990028-c4d9-40d7-b1dc-b45b9f3ccbf2\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j75jm" Oct 30 18:38:18.788704 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.788681 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-j75jm"] Oct 30 18:38:18.793664 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.793644 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-fjdk5"] Oct 30 18:38:18.813552 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.813532 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 30 18:38:18.813552 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.813545 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-lp9z6\"" Oct 30 18:38:18.813694 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.813547 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 30 18:38:18.838653 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.838634 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:18.888057 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.888026 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbjt2\" (UniqueName: \"kubernetes.io/projected/91b2497b-8770-4c64-9b1f-e2ec8744ccf9-kube-api-access-cbjt2\") pod \"migrator-66df5d496b-fjdk5\" (UID: \"91b2497b-8770-4c64-9b1f-e2ec8744ccf9\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fjdk5" Oct 30 18:38:18.888231 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.888214 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:18.888269 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.888254 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh5hg\" (UniqueName: \"kubernetes.io/projected/74990028-c4d9-40d7-b1dc-b45b9f3ccbf2-kube-api-access-xh5hg\") pod \"migrator-66df5d496b-j75jm\" (UID: \"74990028-c4d9-40d7-b1dc-b45b9f3ccbf2\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j75jm" Oct 30 18:38:18.888312 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.888275 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:18.888385 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.888367 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:38:18.888448 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.888435 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls podName:3d5e33b3-69cd-4fc7-8366-14ffa038713e nodeName:}" failed. No retries permitted until 2026-10-30 18:38:19.88841803 +0000 UTC m=+41.602019335 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5bn4v" (UID: "3d5e33b3-69cd-4fc7-8366-14ffa038713e") : secret "insights-runtime-extractor-tls" not found Oct 30 18:38:18.888509 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.888370 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 18:38:18.888509 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.888494 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-rbac-proxy-cm podName:3d5e33b3-69cd-4fc7-8366-14ffa038713e nodeName:}" failed. No retries permitted until 2026-10-30 18:38:19.888481199 +0000 UTC m=+41.602082520 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5bn4v" (UID: "3d5e33b3-69cd-4fc7-8366-14ffa038713e") : configmap "kube-rbac-proxy" not found Oct 30 18:38:18.956540 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.956507 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh5hg\" (UniqueName: \"kubernetes.io/projected/74990028-c4d9-40d7-b1dc-b45b9f3ccbf2-kube-api-access-xh5hg\") pod \"migrator-66df5d496b-j75jm\" (UID: \"74990028-c4d9-40d7-b1dc-b45b9f3ccbf2\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j75jm" Oct 30 18:38:18.958640 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.958613 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:18.988954 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.988926 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbjt2\" (UniqueName: \"kubernetes.io/projected/91b2497b-8770-4c64-9b1f-e2ec8744ccf9-kube-api-access-cbjt2\") pod \"migrator-66df5d496b-fjdk5\" (UID: \"91b2497b-8770-4c64-9b1f-e2ec8744ccf9\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fjdk5" Oct 30 18:38:18.991181 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.991148 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-jwxvn" event={"ID":"51d9053b-abef-45c8-bd1a-67343812dcd9","Type":"ContainerStarted","Data":"f1dfe79d4c2b838b320afd13c56c411338b68785cc9dd0b9c8516c976417f0b0"} Oct 30 18:38:18.992716 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.992700 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 18:38:18.993057 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.993045 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/0.log" Oct 30 18:38:18.993122 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.993073 2578 generic.go:356] "Generic (PLEG): container finished" podID="67b57c01-e4a3-4ae1-ab6e-9b386e181adb" containerID="a69c05894335c282f0d089331d4cc762b19d49ccf1bc60df7aec93b5023cbc6f" exitCode=255 Oct 30 18:38:18.993169 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.993134 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" event={"ID":"67b57c01-e4a3-4ae1-ab6e-9b386e181adb","Type":"ContainerDied","Data":"a69c05894335c282f0d089331d4cc762b19d49ccf1bc60df7aec93b5023cbc6f"} Oct 30 18:38:18.993213 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.993172 2578 scope.go:122] "RemoveContainer" containerID="516be20b73ce53a6212a7e9c428334f320249cb96f8b1990d01a624dca65e269" Oct 30 18:38:18.993356 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.993336 2578 scope.go:122] "RemoveContainer" containerID="a69c05894335c282f0d089331d4cc762b19d49ccf1bc60df7aec93b5023cbc6f" Oct 30 18:38:18.993560 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:18.993538 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-gg799_openshift-console-operator(67b57c01-e4a3-4ae1-ab6e-9b386e181adb)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" podUID="67b57c01-e4a3-4ae1-ab6e-9b386e181adb" Oct 30 18:38:18.994714 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:18.994695 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-95g7n" event={"ID":"cd195fdf-8022-4279-8eca-15d145d59038","Type":"ContainerStarted","Data":"2af4a542f48da14779fd405b3cf22b8fba4e54b68206b4934e780870942015cc"} Oct 30 18:38:19.049554 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.049531 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbjt2\" (UniqueName: \"kubernetes.io/projected/91b2497b-8770-4c64-9b1f-e2ec8744ccf9-kube-api-access-cbjt2\") pod \"migrator-66df5d496b-fjdk5\" (UID: \"91b2497b-8770-4c64-9b1f-e2ec8744ccf9\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fjdk5" Oct 30 18:38:19.094687 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.091677 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j75jm" Oct 30 18:38:19.105551 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.105500 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-6f78d59f99-jwxvn" podStartSLOduration=234.338634298 podStartE2EDuration="4m1.105487015s" totalImagesPullingTime="6.766852717s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:19.104726629 +0000 UTC m=+40.818327944" watchObservedRunningTime="2026-10-30 18:38:19.105487015 +0000 UTC m=+40.819088328" Oct 30 18:38:19.107852 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.107829 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fjdk5" Oct 30 18:38:19.169950 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.168016 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:19.227926 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.227900 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-jmjn6-ghcr6"] Oct 30 18:38:19.334002 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.333960 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-95g7n" podStartSLOduration=34.31395919 podStartE2EDuration="40.333945918s" totalImagesPullingTime="6.019986728s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:19.333334581 +0000 UTC m=+41.046935896" watchObservedRunningTime="2026-10-30 18:38:19.333945918 +0000 UTC m=+41.047547232" Oct 30 18:38:19.470147 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.470110 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-j75jm"] Oct 30 18:38:19.472654 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:38:19.472623 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74990028_c4d9_40d7_b1dc_b45b9f3ccbf2.slice/crio-70efc2349fb099a70d72f8c4332cb650c93cabd5374020a22e3c3ca0c2b662a6 WatchSource:0}: Error finding container 70efc2349fb099a70d72f8c4332cb650c93cabd5374020a22e3c3ca0c2b662a6: Status 404 returned error can't find the container with id 70efc2349fb099a70d72f8c4332cb650c93cabd5374020a22e3c3ca0c2b662a6 Oct 30 18:38:19.506678 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.506654 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-fjdk5"] Oct 30 18:38:19.508735 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:38:19.508713 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91b2497b_8770_4c64_9b1f_e2ec8744ccf9.slice/crio-06192cb253d0f89797abd9283cd05e09470c34b4bc601a192fd03ad567b03272 WatchSource:0}: Error finding container 06192cb253d0f89797abd9283cd05e09470c34b4bc601a192fd03ad567b03272: Status 404 returned error can't find the container with id 06192cb253d0f89797abd9283cd05e09470c34b4bc601a192fd03ad567b03272 Oct 30 18:38:19.787874 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.787841 2578 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-t2jzr"] Oct 30 18:38:19.818222 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.818197 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" Oct 30 18:38:19.847290 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.847265 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-t2jzr"] Oct 30 18:38:19.873136 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.873110 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Oct 30 18:38:19.873267 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.873219 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Oct 30 18:38:19.873267 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.873235 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-b7gws\"" Oct 30 18:38:19.873267 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.873222 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Oct 30 18:38:19.873267 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.873254 2578 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Oct 30 18:38:19.904004 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.903981 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:19.904149 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.904020 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca-signing-key\") pod \"service-ca-5fffc97bdf-t2jzr\" (UID: \"77f8df88-fa80-42a5-9ac6-33b2fea5d7ca\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" Oct 30 18:38:19.904149 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.904037 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca-signing-cabundle\") pod \"service-ca-5fffc97bdf-t2jzr\" (UID: \"77f8df88-fa80-42a5-9ac6-33b2fea5d7ca\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" Oct 30 18:38:19.904149 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.904058 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:19.904255 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.904167 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6f6l\" (UniqueName: \"kubernetes.io/projected/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca-kube-api-access-b6f6l\") pod \"service-ca-5fffc97bdf-t2jzr\" (UID: \"77f8df88-fa80-42a5-9ac6-33b2fea5d7ca\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" Oct 30 18:38:19.904293 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:19.904268 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:38:19.904322 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:19.904317 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls podName:3d5e33b3-69cd-4fc7-8366-14ffa038713e nodeName:}" failed. No retries permitted until 2026-10-30 18:38:21.90430599 +0000 UTC m=+43.617907285 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5bn4v" (UID: "3d5e33b3-69cd-4fc7-8366-14ffa038713e") : secret "insights-runtime-extractor-tls" not found Oct 30 18:38:19.904378 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:19.904360 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Oct 30 18:38:19.904426 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:19.904416 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-rbac-proxy-cm podName:3d5e33b3-69cd-4fc7-8366-14ffa038713e nodeName:}" failed. No retries permitted until 2026-10-30 18:38:21.904402124 +0000 UTC m=+43.618003417 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5bn4v" (UID: "3d5e33b3-69cd-4fc7-8366-14ffa038713e") : configmap "kube-rbac-proxy" not found Oct 30 18:38:19.999175 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.999140 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" event={"ID":"290aeefa-831a-4e2d-bc4c-11defdfee2d0","Type":"ContainerStarted","Data":"a7014b135fd44083e8acfe64076a984ac9caf501ddd92d5875d7923cebd3023d"} Oct 30 18:38:19.999635 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:19.999202 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" event={"ID":"290aeefa-831a-4e2d-bc4c-11defdfee2d0","Type":"ContainerStarted","Data":"ff0ec68a90647ce926cdb4307ff2be59548cfbdc73a2e046c61eade4003464e8"} Oct 30 18:38:20.000405 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:20.000376 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fjdk5" event={"ID":"91b2497b-8770-4c64-9b1f-e2ec8744ccf9","Type":"ContainerStarted","Data":"06192cb253d0f89797abd9283cd05e09470c34b4bc601a192fd03ad567b03272"} Oct 30 18:38:20.001591 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:20.001555 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j75jm" event={"ID":"74990028-c4d9-40d7-b1dc-b45b9f3ccbf2","Type":"ContainerStarted","Data":"70efc2349fb099a70d72f8c4332cb650c93cabd5374020a22e3c3ca0c2b662a6"} Oct 30 18:38:20.005597 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:20.003323 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 18:38:20.005597 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:20.004094 2578 scope.go:122] "RemoveContainer" containerID="a69c05894335c282f0d089331d4cc762b19d49ccf1bc60df7aec93b5023cbc6f" Oct 30 18:38:20.005597 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:20.004264 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-gg799_openshift-console-operator(67b57c01-e4a3-4ae1-ab6e-9b386e181adb)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" podUID="67b57c01-e4a3-4ae1-ab6e-9b386e181adb" Oct 30 18:38:20.005597 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:20.004616 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca-signing-key\") pod \"service-ca-5fffc97bdf-t2jzr\" (UID: \"77f8df88-fa80-42a5-9ac6-33b2fea5d7ca\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" Oct 30 18:38:20.005597 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:20.004644 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca-signing-cabundle\") pod \"service-ca-5fffc97bdf-t2jzr\" (UID: \"77f8df88-fa80-42a5-9ac6-33b2fea5d7ca\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" Oct 30 18:38:20.005597 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:20.004718 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6f6l\" (UniqueName: \"kubernetes.io/projected/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca-kube-api-access-b6f6l\") pod \"service-ca-5fffc97bdf-t2jzr\" (UID: \"77f8df88-fa80-42a5-9ac6-33b2fea5d7ca\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" Oct 30 18:38:20.005597 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:20.005500 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:38:20.006235 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:20.006215 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca-signing-cabundle\") pod \"service-ca-5fffc97bdf-t2jzr\" (UID: \"77f8df88-fa80-42a5-9ac6-33b2fea5d7ca\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" Oct 30 18:38:20.007853 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:20.007835 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca-signing-key\") pod \"service-ca-5fffc97bdf-t2jzr\" (UID: \"77f8df88-fa80-42a5-9ac6-33b2fea5d7ca\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" Oct 30 18:38:20.049588 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:20.049512 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:20.061224 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:20.061200 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6f6l\" (UniqueName: \"kubernetes.io/projected/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca-kube-api-access-b6f6l\") pod \"service-ca-5fffc97bdf-t2jzr\" (UID: \"77f8df88-fa80-42a5-9ac6-33b2fea5d7ca\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" Oct 30 18:38:20.113953 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:20.113907 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" podStartSLOduration=2.113894484 podStartE2EDuration="2.113894484s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:20.112601421 +0000 UTC m=+41.826202734" watchObservedRunningTime="2026-10-30 18:38:20.113894484 +0000 UTC m=+41.827495797" Oct 30 18:38:20.142455 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:20.142429 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" Oct 30 18:38:20.247459 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:20.247431 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:20.483387 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:20.483310 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:20.534046 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:20.534024 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-5fffc97bdf-t2jzr"] Oct 30 18:38:20.536988 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:38:20.536954 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77f8df88_fa80_42a5_9ac6_33b2fea5d7ca.slice/crio-b7ce03394c927b190fb2b2d52042e55bddcbb68628f97ae7b932e5e2266f68d3 WatchSource:0}: Error finding container b7ce03394c927b190fb2b2d52042e55bddcbb68628f97ae7b932e5e2266f68d3: Status 404 returned error can't find the container with id b7ce03394c927b190fb2b2d52042e55bddcbb68628f97ae7b932e5e2266f68d3 Oct 30 18:38:20.742625 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:20.742553 2578 kubelet.go:2727] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:20.803004 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:20.802976 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:21.016889 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:21.016801 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" event={"ID":"77f8df88-fa80-42a5-9ac6-33b2fea5d7ca","Type":"ContainerStarted","Data":"81c461b0617af805b73d72a2b63c2b67e55d8615c0812fa8fce68136d143ad64"} Oct 30 18:38:21.017315 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:21.016907 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" event={"ID":"77f8df88-fa80-42a5-9ac6-33b2fea5d7ca","Type":"ContainerStarted","Data":"b7ce03394c927b190fb2b2d52042e55bddcbb68628f97ae7b932e5e2266f68d3"} Oct 30 18:38:21.017974 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:21.017950 2578 scope.go:122] "RemoveContainer" containerID="a69c05894335c282f0d089331d4cc762b19d49ccf1bc60df7aec93b5023cbc6f" Oct 30 18:38:21.018185 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:21.018161 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-gg799_openshift-console-operator(67b57c01-e4a3-4ae1-ab6e-9b386e181adb)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" podUID="67b57c01-e4a3-4ae1-ab6e-9b386e181adb" Oct 30 18:38:21.019188 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:21.019163 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:21.133339 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:21.133278 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" podStartSLOduration=2.13325768 podStartE2EDuration="2.13325768s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:21.131245564 +0000 UTC m=+42.844846879" watchObservedRunningTime="2026-10-30 18:38:21.13325768 +0000 UTC m=+42.846858997" Oct 30 18:38:21.844673 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:21.844637 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:21.922086 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:21.922056 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:21.922237 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:21.922103 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:21.922237 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:21.922195 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:38:21.922302 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:21.922258 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls podName:3d5e33b3-69cd-4fc7-8366-14ffa038713e nodeName:}" failed. No retries permitted until 2026-10-30 18:38:25.92223991 +0000 UTC m=+47.635841203 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5bn4v" (UID: "3d5e33b3-69cd-4fc7-8366-14ffa038713e") : secret "insights-runtime-extractor-tls" not found Oct 30 18:38:21.922566 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:21.922551 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:22.022737 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:22.022606 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fjdk5" event={"ID":"91b2497b-8770-4c64-9b1f-e2ec8744ccf9","Type":"ContainerStarted","Data":"b6184a17c46caf5e402c7230a3bdb06c878f748f3fc75bbafc901cc0e834f7e2"} Oct 30 18:38:22.025542 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:22.025517 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j75jm" event={"ID":"74990028-c4d9-40d7-b1dc-b45b9f3ccbf2","Type":"ContainerStarted","Data":"301df9cd47ea406d19d72822c15cbbb40e817553fb4b9be187ffd76421ef2982"} Oct 30 18:38:22.136274 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:22.136226 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fjdk5" podStartSLOduration=1.93161192 podStartE2EDuration="4.136210713s" totalImagesPullingTime="2.204598793s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:22.135247014 +0000 UTC m=+43.848848328" watchObservedRunningTime="2026-10-30 18:38:22.136210713 +0000 UTC m=+43.849812028" Oct 30 18:38:22.179590 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:22.179560 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5mmnd_2bfeb3c6-d71a-4b5e-96db-198da488f7a9/dns-node-resolver/0.log" Oct 30 18:38:22.250117 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:22.250075 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j75jm" podStartSLOduration=2.215941033 podStartE2EDuration="4.250061075s" totalImagesPullingTime="2.034120042s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:22.248878 +0000 UTC m=+43.962479313" watchObservedRunningTime="2026-10-30 18:38:22.250061075 +0000 UTC m=+43.963662389" Oct 30 18:38:22.313381 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:22.313306 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:22.493934 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:22.493895 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:23.186172 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:23.186136 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:23.379880 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:23.379851 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-kjn4r_15f5b642-cd06-42ad-9832-dcfe438ea4e5/node-ca/0.log" Oct 30 18:38:23.414049 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:23.414024 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:23.782803 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:23.782765 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:23.982428 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:23.982404 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fdrkw_63658d25-6d63-42e2-9755-8ca66a322d61/serve-healthcheck-canary/0.log" Oct 30 18:38:23.999129 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:23.999101 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:24.167194 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:24.167166 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:24.179696 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:24.179675 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-fjdk5_91b2497b-8770-4c64-9b1f-e2ec8744ccf9/migrator/0.log" Oct 30 18:38:24.380987 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:24.380959 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-j75jm_74990028-c4d9-40d7-b1dc-b45b9f3ccbf2/migrator/0.log" Oct 30 18:38:24.580306 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:24.580233 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7d4bc45dd6-689xp_89271a7f-e9c4-43dd-a320-54900b3fb8b7/kube-storage-version-migrator-operator/0.log" Oct 30 18:38:24.590714 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:24.590693 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:25.182735 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:25.182694 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:25.386283 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:25.386249 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:25.783445 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:25.783411 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:25.958538 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:25.958504 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:25.958724 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:25.958675 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Oct 30 18:38:25.958778 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:25.958739 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls podName:3d5e33b3-69cd-4fc7-8366-14ffa038713e nodeName:}" failed. No retries permitted until 2026-10-30 18:38:33.95872502 +0000 UTC m=+55.672326315 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5bn4v" (UID: "3d5e33b3-69cd-4fc7-8366-14ffa038713e") : secret "insights-runtime-extractor-tls" not found Oct 30 18:38:25.988674 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:25.988632 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:26.160443 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.160411 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:26.162796 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.162772 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:26.213491 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.213465 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:26.261144 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.261109 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hqzbs\" (UID: \"c135952e-12c3-4a65-bd3d-0d0b1aba49d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" Oct 30 18:38:26.262167 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.262143 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:26.262355 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.262323 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:26.262503 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.262487 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:26.262650 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.262634 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:26.262803 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.262782 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:26.263120 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:26.263102 2578 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Oct 30 18:38:26.263618 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.263104 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:26.263717 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:26.263197 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:38:26.263717 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:26.263682 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7dbbfc9475-s4vg9: secret "image-registry-tls" not found Oct 30 18:38:26.263831 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:26.263254 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Oct 30 18:38:26.263831 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:26.263732 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f6754678-55bz6: secret "image-registry-tls" not found Oct 30 18:38:26.263831 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:26.263798 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls podName:872d91f9-1bbc-4db1-bffe-6d6241175fee nodeName:}" failed. No retries permitted until 2026-10-30 18:38:42.263778905 +0000 UTC m=+63.977380203 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls") pod "image-registry-7dbbfc9475-s4vg9" (UID: "872d91f9-1bbc-4db1-bffe-6d6241175fee") : secret "image-registry-tls" not found Oct 30 18:38:26.263831 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:26.263816 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls podName:4a88dde4-2103-476d-81c4-4b255cff4828 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:42.263806076 +0000 UTC m=+63.977407371 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls") pod "image-registry-7f6754678-55bz6" (UID: "4a88dde4-2103-476d-81c4-4b255cff4828") : secret "image-registry-tls" not found Oct 30 18:38:26.264099 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:26.264069 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls podName:9cd97653-a745-4fd6-9a6c-71aaa742b415 nodeName:}" failed. No retries permitted until 2026-10-30 18:38:42.264046603 +0000 UTC m=+63.977647913 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls") pod "cluster-samples-operator-6ddb6c8b94-wf7rb" (UID: "9cd97653-a745-4fd6-9a6c-71aaa742b415") : secret "samples-operator-tls" not found Oct 30 18:38:26.265044 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.265018 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hqzbs\" (UID: \"c135952e-12c3-4a65-bd3d-0d0b1aba49d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" Oct 30 18:38:26.266018 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.265996 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:26.273047 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.272775 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:26.313948 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.312354 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" Oct 30 18:38:26.365449 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.365420 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:26.365648 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:26.365624 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Oct 30 18:38:26.365712 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:26.365698 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls podName:a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f nodeName:}" failed. No retries permitted until 2026-10-30 18:38:42.365677373 +0000 UTC m=+64.079278683 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls") pod "dns-default-kkzsh" (UID: "a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f") : secret "dns-default-metrics-tls" not found Oct 30 18:38:26.593406 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.593338 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:26.626527 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.626498 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-884576c57-td46q"] Oct 30 18:38:26.669790 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:38:26.669766 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2797391b_fd3d_4222_82c8_b28a2d16c9e3.slice/crio-1d9056edc0e80f4fa89a98a908130771563b58b346ba3d202690e1907398462f WatchSource:0}: Error finding container 1d9056edc0e80f4fa89a98a908130771563b58b346ba3d202690e1907398462f: Status 404 returned error can't find the container with id 1d9056edc0e80f4fa89a98a908130771563b58b346ba3d202690e1907398462f Oct 30 18:38:26.680605 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.680566 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-797b6dc769-r2cdl"] Oct 30 18:38:26.682822 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:38:26.682799 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3180fff2_f289_47e6_a37c_34c5dff50aa2.slice/crio-5dd77f7e140b9dab12799c0b35a5dc1c3df7894456b2c54abf57f224455ee922 WatchSource:0}: Error finding container 5dd77f7e140b9dab12799c0b35a5dc1c3df7894456b2c54abf57f224455ee922: Status 404 returned error can't find the container with id 5dd77f7e140b9dab12799c0b35a5dc1c3df7894456b2c54abf57f224455ee922 Oct 30 18:38:26.729187 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:38:26.729162 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc135952e_12c3_4a65_bd3d_0d0b1aba49d1.slice/crio-d11cbe37ba26264691e609ca8fc29e574c0b34e9a14cc855094fdc0350d02859 WatchSource:0}: Error finding container d11cbe37ba26264691e609ca8fc29e574c0b34e9a14cc855094fdc0350d02859: Status 404 returned error can't find the container with id d11cbe37ba26264691e609ca8fc29e574c0b34e9a14cc855094fdc0350d02859 Oct 30 18:38:26.729766 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.729747 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-86c478bb88-hqzbs"] Oct 30 18:38:26.968964 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.968933 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:26.969357 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:26.969339 2578 scope.go:122] "RemoveContainer" containerID="a69c05894335c282f0d089331d4cc762b19d49ccf1bc60df7aec93b5023cbc6f" Oct 30 18:38:26.969599 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:26.969549 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7cd56b8dbc-gg799_openshift-console-operator(67b57c01-e4a3-4ae1-ab6e-9b386e181adb)\"" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" podUID="67b57c01-e4a3-4ae1-ab6e-9b386e181adb" Oct 30 18:38:27.041194 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:27.041161 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" event={"ID":"2797391b-fd3d-4222-82c8-b28a2d16c9e3","Type":"ContainerStarted","Data":"1d9056edc0e80f4fa89a98a908130771563b58b346ba3d202690e1907398462f"} Oct 30 18:38:27.042648 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:27.042620 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-797b6dc769-r2cdl" event={"ID":"3180fff2-f289-47e6-a37c-34c5dff50aa2","Type":"ContainerStarted","Data":"f07eea8a77b71372995e62fbf80918532ea942903f11f6c2a459859ace60e4a3"} Oct 30 18:38:27.042768 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:27.042668 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-797b6dc769-r2cdl" event={"ID":"3180fff2-f289-47e6-a37c-34c5dff50aa2","Type":"ContainerStarted","Data":"5dd77f7e140b9dab12799c0b35a5dc1c3df7894456b2c54abf57f224455ee922"} Oct 30 18:38:27.043806 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:27.043787 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" event={"ID":"c135952e-12c3-4a65-bd3d-0d0b1aba49d1","Type":"ContainerStarted","Data":"d11cbe37ba26264691e609ca8fc29e574c0b34e9a14cc855094fdc0350d02859"} Oct 30 18:38:27.159959 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:27.159912 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/router-default-797b6dc769-r2cdl" podStartSLOduration=252.159898044 podStartE2EDuration="4m12.159898044s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:27.15833518 +0000 UTC m=+48.871936484" watchObservedRunningTime="2026-10-30 18:38:27.159898044 +0000 UTC m=+48.873499357" Oct 30 18:38:27.189397 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:27.189364 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:27.273536 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:27.273455 2578 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:27.276080 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:27.276058 2578 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:27.387539 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:27.387506 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:27.556764 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:27.556690 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:27.786105 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:27.786053 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:27.981665 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:27.981628 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:28.048030 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:28.047999 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:28.049730 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:28.049706 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:28.589891 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:28.589848 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:28.717500 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:28.717429 2578 scope.go:122] "RemoveContainer" containerID="a69c05894335c282f0d089331d4cc762b19d49ccf1bc60df7aec93b5023cbc6f" Oct 30 18:38:29.052804 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:29.052726 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 18:38:29.053240 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:29.052802 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" event={"ID":"67b57c01-e4a3-4ae1-ab6e-9b386e181adb","Type":"ContainerStarted","Data":"c60015a3d2d7d8cc567cddb04d243a2e60e10b4302fee9dfb2084ed1ca4b290b"} Oct 30 18:38:29.053240 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:29.053132 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:29.054189 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:29.054160 2578 patch_prober.go:29] interesting pod/console-operator-7cd56b8dbc-gg799 container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.132.0.17:8443/readyz\": dial tcp 10.132.0.17:8443: connect: connection refused" start-of-body= Oct 30 18:38:29.054301 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:29.054248 2578 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" podUID="67b57c01-e4a3-4ae1-ab6e-9b386e181adb" containerName="console-operator" probeResult="failure" output="Get \"https://10.132.0.17:8443/readyz\": dial tcp 10.132.0.17:8443: connect: connection refused" Oct 30 18:38:29.054418 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:29.054386 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" event={"ID":"2797391b-fd3d-4222-82c8-b28a2d16c9e3","Type":"ContainerStarted","Data":"411741f4c152fb4ee05ec1614df88302664658bd236ac3ba1e31d04ba15f6c53"} Oct 30 18:38:29.055799 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:29.055779 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" event={"ID":"c135952e-12c3-4a65-bd3d-0d0b1aba49d1","Type":"ContainerStarted","Data":"2ffd1afc183dc4554dfdb2371c229c4c031adcdb6e77b42b95199499b19db21d"} Oct 30 18:38:29.190401 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:29.190359 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:29.286010 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:29.285961 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" podStartSLOduration=252.139977266 podStartE2EDuration="4m14.28594663s" totalImagesPullingTime="2.145969364s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:29.28506028 +0000 UTC m=+50.998661595" watchObservedRunningTime="2026-10-30 18:38:29.28594663 +0000 UTC m=+50.999547944" Oct 30 18:38:29.398597 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:29.398355 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" podStartSLOduration=216.702953285 podStartE2EDuration="3m38.398310916s" totalImagesPullingTime="1.695357631s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:29.398101825 +0000 UTC m=+51.111703140" watchObservedRunningTime="2026-10-30 18:38:29.398310916 +0000 UTC m=+51.111912231" Oct 30 18:38:29.410107 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:29.410079 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:29.788487 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:29.788420 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:30.007407 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:30.007368 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:30.063949 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:30.063878 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:30.179543 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:30.179497 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:30.379294 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:30.379220 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:30.556862 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:30.556825 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:30.735693 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:30.735658 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:31.935954 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:31.935919 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:32.105915 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:32.105879 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:32.524789 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:32.524756 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:32.751348 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:32.751315 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:32.985390 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:32.985342 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:33.183627 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:33.183565 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:33.386730 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:33.386651 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:33.914551 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:33.914524 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:34.036173 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:34.036142 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:34.038448 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:34.038427 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:34.222298 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:34.222222 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:34.591490 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:34.591466 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5bn4v"] Oct 30 18:38:34.593911 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:38:34.593886 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d5e33b3_69cd_4fc7_8366_14ffa038713e.slice/crio-4336e440d847cb102b5a84cf5db4e4868879b7a45b75ad04742b4b0303ca407f WatchSource:0}: Error finding container 4336e440d847cb102b5a84cf5db4e4868879b7a45b75ad04742b4b0303ca407f: Status 404 returned error can't find the container with id 4336e440d847cb102b5a84cf5db4e4868879b7a45b75ad04742b4b0303ca407f Oct 30 18:38:35.077600 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:35.077554 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5bn4v" event={"ID":"3d5e33b3-69cd-4fc7-8366-14ffa038713e","Type":"ContainerStarted","Data":"fe08a2336e9e0a16656d24db1f217dcc922bca819e1c2ab273289665ea95035a"} Oct 30 18:38:35.077967 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:35.077620 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5bn4v" event={"ID":"3d5e33b3-69cd-4fc7-8366-14ffa038713e","Type":"ContainerStarted","Data":"4336e440d847cb102b5a84cf5db4e4868879b7a45b75ad04742b4b0303ca407f"} Oct 30 18:38:36.083370 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:36.083293 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5bn4v" event={"ID":"3d5e33b3-69cd-4fc7-8366-14ffa038713e","Type":"ContainerStarted","Data":"3f579516d16e959ed05a163038255faf8cde2078b257cd1992f2ef973b76d946"} Oct 30 18:38:37.329328 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:37.329295 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:37.508748 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:37.508708 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:38.092078 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:38.092039 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5bn4v" event={"ID":"3d5e33b3-69cd-4fc7-8366-14ffa038713e","Type":"ContainerStarted","Data":"7235857c0f8045c49454bf7e2db0ab767dc60cbdd6c51a1fab8906ff091be9b2"} Oct 30 18:38:38.255921 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:38.255873 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5bn4v" podStartSLOduration=17.323486636 podStartE2EDuration="20.255859539s" totalImagesPullingTime="2.932372903s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 18:38:18 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:38.20538106 +0000 UTC m=+59.918982375" watchObservedRunningTime="2026-10-30 18:38:38.255859539 +0000 UTC m=+59.969460852" Oct 30 18:38:40.824906 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:40.824873 2578 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7dbbfc9475-s4vg9"] Oct 30 18:38:40.825476 ip-10-0-132-122 kubenswrapper[2578]: E1030 18:38:40.825059 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" podUID="872d91f9-1bbc-4db1-bffe-6d6241175fee" Oct 30 18:38:40.997998 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:40.997962 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:41.101556 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.101481 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:41.107313 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.107287 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:41.196134 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.196106 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/872d91f9-1bbc-4db1-bffe-6d6241175fee-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/872d91f9-1bbc-4db1-bffe-6d6241175fee-image-registry-private-configuration\") pod \"872d91f9-1bbc-4db1-bffe-6d6241175fee\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " Oct 30 18:38:41.196307 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.196141 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-kube-api-access-cst78\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-kube-api-access-cst78\") pod \"872d91f9-1bbc-4db1-bffe-6d6241175fee\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " Oct 30 18:38:41.196307 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.196183 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/872d91f9-1bbc-4db1-bffe-6d6241175fee-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/872d91f9-1bbc-4db1-bffe-6d6241175fee-ca-trust-extracted\") pod \"872d91f9-1bbc-4db1-bffe-6d6241175fee\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " Oct 30 18:38:41.196307 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.196224 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/872d91f9-1bbc-4db1-bffe-6d6241175fee-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/872d91f9-1bbc-4db1-bffe-6d6241175fee-trusted-ca\") pod \"872d91f9-1bbc-4db1-bffe-6d6241175fee\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " Oct 30 18:38:41.196307 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.196253 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/872d91f9-1bbc-4db1-bffe-6d6241175fee-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/872d91f9-1bbc-4db1-bffe-6d6241175fee-installation-pull-secrets\") pod \"872d91f9-1bbc-4db1-bffe-6d6241175fee\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " Oct 30 18:38:41.196307 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.196309 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-bound-sa-token\") pod \"872d91f9-1bbc-4db1-bffe-6d6241175fee\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " Oct 30 18:38:41.196559 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.196342 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-certificates\") pod \"872d91f9-1bbc-4db1-bffe-6d6241175fee\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " Oct 30 18:38:41.196559 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.196525 2578 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/872d91f9-1bbc-4db1-bffe-6d6241175fee-ca-trust-extracted" pod "872d91f9-1bbc-4db1-bffe-6d6241175fee" (UID: "872d91f9-1bbc-4db1-bffe-6d6241175fee"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:38:41.196733 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.196711 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/872d91f9-1bbc-4db1-bffe-6d6241175fee-ca-trust-extracted\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:38:41.196902 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.196746 2578 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/872d91f9-1bbc-4db1-bffe-6d6241175fee-trusted-ca" pod "872d91f9-1bbc-4db1-bffe-6d6241175fee" (UID: "872d91f9-1bbc-4db1-bffe-6d6241175fee"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:41.196985 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.196792 2578 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-certificates" pod "872d91f9-1bbc-4db1-bffe-6d6241175fee" (UID: "872d91f9-1bbc-4db1-bffe-6d6241175fee"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:38:41.198480 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.198452 2578 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/872d91f9-1bbc-4db1-bffe-6d6241175fee-installation-pull-secrets" pod "872d91f9-1bbc-4db1-bffe-6d6241175fee" (UID: "872d91f9-1bbc-4db1-bffe-6d6241175fee"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:41.198639 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.198475 2578 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-kube-api-access-cst78" pod "872d91f9-1bbc-4db1-bffe-6d6241175fee" (UID: "872d91f9-1bbc-4db1-bffe-6d6241175fee"). InnerVolumeSpecName "kube-api-access-cst78". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:38:41.198639 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.198502 2578 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/872d91f9-1bbc-4db1-bffe-6d6241175fee-image-registry-private-configuration" pod "872d91f9-1bbc-4db1-bffe-6d6241175fee" (UID: "872d91f9-1bbc-4db1-bffe-6d6241175fee"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:38:41.198639 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.198528 2578 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-bound-sa-token" pod "872d91f9-1bbc-4db1-bffe-6d6241175fee" (UID: "872d91f9-1bbc-4db1-bffe-6d6241175fee"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:38:41.232001 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.231968 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:41.297422 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.297396 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/872d91f9-1bbc-4db1-bffe-6d6241175fee-installation-pull-secrets\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:38:41.297422 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.297419 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-bound-sa-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:38:41.297559 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.297429 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-certificates\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:38:41.297559 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.297439 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/872d91f9-1bbc-4db1-bffe-6d6241175fee-image-registry-private-configuration\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:38:41.297559 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.297448 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cst78\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-kube-api-access-cst78\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:38:41.297559 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.297456 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/872d91f9-1bbc-4db1-bffe-6d6241175fee-trusted-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:38:41.402177 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.402144 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:41.586785 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.586750 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:41.776668 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:41.776590 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:42.105264 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.105192 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:42.305170 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.305140 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:42.305336 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.305177 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:42.305336 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.305206 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:42.307650 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.307619 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:42.307650 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.307642 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls\") pod \"image-registry-7dbbfc9475-s4vg9\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " pod="openshift-image-registry/image-registry-7dbbfc9475-s4vg9" Oct 30 18:38:42.307794 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.307733 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:42.364660 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.364600 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2529f\"" Oct 30 18:38:42.373649 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.373631 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:42.384740 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.384709 2578 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7dbbfc9475-s4vg9"] Oct 30 18:38:42.388236 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.388212 2578 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7dbbfc9475-s4vg9"] Oct 30 18:38:42.406184 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.406160 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls\") pod \"872d91f9-1bbc-4db1-bffe-6d6241175fee\" (UID: \"872d91f9-1bbc-4db1-bffe-6d6241175fee\") " Oct 30 18:38:42.406339 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.406325 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:42.408082 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.408055 2578 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls" pod "872d91f9-1bbc-4db1-bffe-6d6241175fee" (UID: "872d91f9-1bbc-4db1-bffe-6d6241175fee"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:38:42.408453 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.408434 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:42.507362 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.507327 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/872d91f9-1bbc-4db1-bffe-6d6241175fee-registry-tls\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:38:42.539327 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.539299 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-m9jbf\"" Oct 30 18:38:42.548400 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.548379 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:42.647241 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.647218 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xhgds\"" Oct 30 18:38:42.656356 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.656333 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:42.723568 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.723365 2578 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="872d91f9-1bbc-4db1-bffe-6d6241175fee" path="/var/lib/kubelet/pods/872d91f9-1bbc-4db1-bffe-6d6241175fee/volumes" Oct 30 18:38:42.797288 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.797262 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f6754678-55bz6"] Oct 30 18:38:42.800140 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:38:42.800112 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a88dde4_2103_476d_81c4_4b255cff4828.slice/crio-4971600d17965249e566952e3d5de9e09491c9d11640b0f63933dc353ea1ab09 WatchSource:0}: Error finding container 4971600d17965249e566952e3d5de9e09491c9d11640b0f63933dc353ea1ab09: Status 404 returned error can't find the container with id 4971600d17965249e566952e3d5de9e09491c9d11640b0f63933dc353ea1ab09 Oct 30 18:38:42.913199 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:42.913051 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb"] Oct 30 18:38:43.022673 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:43.022651 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-kkzsh"] Oct 30 18:38:43.025764 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:38:43.025735 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda2195bb6_ac20_4e5f_b3e9_69b6e5b9b98f.slice/crio-e5622c316fa0601a6308ab7fced872c54601ccf1c0f8b094ac3870a49d6fc09e WatchSource:0}: Error finding container e5622c316fa0601a6308ab7fced872c54601ccf1c0f8b094ac3870a49d6fc09e: Status 404 returned error can't find the container with id e5622c316fa0601a6308ab7fced872c54601ccf1c0f8b094ac3870a49d6fc09e Oct 30 18:38:43.111061 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:43.111023 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f6754678-55bz6" event={"ID":"4a88dde4-2103-476d-81c4-4b255cff4828","Type":"ContainerStarted","Data":"1f208f81f72a5c903471774bf4293139af22fff813196ac6c473474aebec507d"} Oct 30 18:38:43.111499 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:43.111088 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f6754678-55bz6" event={"ID":"4a88dde4-2103-476d-81c4-4b255cff4828","Type":"ContainerStarted","Data":"4971600d17965249e566952e3d5de9e09491c9d11640b0f63933dc353ea1ab09"} Oct 30 18:38:43.111499 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:43.111121 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:43.112351 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:43.112325 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" event={"ID":"9cd97653-a745-4fd6-9a6c-71aaa742b415","Type":"ContainerStarted","Data":"12afdaecda545f8c43bef504323a532ac5a46dbabd623069abfdac6501c84655"} Oct 30 18:38:43.113500 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:43.113479 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kkzsh" event={"ID":"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f","Type":"ContainerStarted","Data":"e5622c316fa0601a6308ab7fced872c54601ccf1c0f8b094ac3870a49d6fc09e"} Oct 30 18:38:43.228251 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:43.228158 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7f6754678-55bz6" podStartSLOduration=56.228143494 podStartE2EDuration="56.228143494s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:43.227153946 +0000 UTC m=+64.940755275" watchObservedRunningTime="2026-10-30 18:38:43.228143494 +0000 UTC m=+64.941744807" Oct 30 18:38:43.717256 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:43.717222 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:38:43.717437 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:43.717264 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs\") pod \"network-metrics-daemon-2s5rm\" (UID: \"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2\") " pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:38:43.768682 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:43.768651 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 18:38:43.768793 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:43.768743 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 18:38:43.780133 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:43.780102 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:38:43.780236 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:43.780131 2578 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs\") pod \"network-metrics-daemon-2s5rm\" (UID: \"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2\") " pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:38:43.915995 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:43.915965 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6nrlk\"" Oct 30 18:38:43.925209 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:43.925188 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:38:43.929726 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:43.929674 2578 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-cts7j\"" Oct 30 18:38:43.939673 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:43.939651 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:38:44.301648 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:44.301503 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-2s5rm"] Oct 30 18:38:44.352014 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:44.351985 2578 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-hmzzs"] Oct 30 18:38:44.352422 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:38:44.352391 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d3a00e6_202e_4ffb_b8e0_67f15d831ae2.slice/crio-6898926c00a6bae4a7eaa36bbcf4186056d4fdeeb32b72ad64de694e4ff305b3 WatchSource:0}: Error finding container 6898926c00a6bae4a7eaa36bbcf4186056d4fdeeb32b72ad64de694e4ff305b3: Status 404 returned error can't find the container with id 6898926c00a6bae4a7eaa36bbcf4186056d4fdeeb32b72ad64de694e4ff305b3 Oct 30 18:38:44.356254 ip-10-0-132-122 kubenswrapper[2578]: W1030 18:38:44.356205 2578 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c9b04e2_b099_4704_8786_b64a8991d0f4.slice/crio-ba4f4ec128c62434c0766741f1006e473a81b720369bc4ea6e3a8d1cadcb6c31 WatchSource:0}: Error finding container ba4f4ec128c62434c0766741f1006e473a81b720369bc4ea6e3a8d1cadcb6c31: Status 404 returned error can't find the container with id ba4f4ec128c62434c0766741f1006e473a81b720369bc4ea6e3a8d1cadcb6c31 Oct 30 18:38:45.124494 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:45.124459 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-hmzzs" event={"ID":"8c9b04e2-b099-4704-8786-b64a8991d0f4","Type":"ContainerStarted","Data":"ba4f4ec128c62434c0766741f1006e473a81b720369bc4ea6e3a8d1cadcb6c31"} Oct 30 18:38:45.125885 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:45.125852 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2s5rm" event={"ID":"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2","Type":"ContainerStarted","Data":"6898926c00a6bae4a7eaa36bbcf4186056d4fdeeb32b72ad64de694e4ff305b3"} Oct 30 18:38:45.403377 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:45.403290 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:45.629451 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:45.629364 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:46.132875 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:46.132835 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" event={"ID":"9cd97653-a745-4fd6-9a6c-71aaa742b415","Type":"ContainerStarted","Data":"37583ed7d8bb304f91c5d49860e743e88de77357cab25d96e47a34e2148d71d2"} Oct 30 18:38:46.133057 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:46.132912 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" event={"ID":"9cd97653-a745-4fd6-9a6c-71aaa742b415","Type":"ContainerStarted","Data":"1072e3699400b08f9b8915d5bd4821c4a867b06a140171452772d2f9696b7f71"} Oct 30 18:38:46.135020 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:46.134992 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kkzsh" event={"ID":"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f","Type":"ContainerStarted","Data":"ff837327947816e58c66012912eb75cd3cdefc060b61666e6f0ee58611000f4a"} Oct 30 18:38:46.135135 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:46.135038 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-kkzsh" event={"ID":"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f","Type":"ContainerStarted","Data":"81c0aaf2564f67ed520d1b299e79da28f2e6509b9cfb1887932b4b67777d8988"} Oct 30 18:38:46.135135 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:46.135121 2578 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:46.154252 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:46.154220 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:46.247657 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:46.247585 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" podStartSLOduration=269.065198646 podStartE2EDuration="4m31.247556948s" totalImagesPullingTime="2.182358302s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:34:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:46.246744471 +0000 UTC m=+67.960345785" watchObservedRunningTime="2026-10-30 18:38:46.247556948 +0000 UTC m=+67.961158261" Oct 30 18:38:46.367374 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:46.367313 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-kkzsh" podStartSLOduration=34.102999708 podStartE2EDuration="36.367293925s" totalImagesPullingTime="2.264294217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:38:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:46.364822711 +0000 UTC m=+68.078424027" watchObservedRunningTime="2026-10-30 18:38:46.367293925 +0000 UTC m=+68.080895239" Oct 30 18:38:47.141363 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:47.141318 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2s5rm" event={"ID":"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2","Type":"ContainerStarted","Data":"8a794f077e5c6ee63c89f199ac9123df77e614027308d029853e52f0f8d4b6d9"} Oct 30 18:38:47.141821 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:47.141378 2578 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-2s5rm" event={"ID":"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2","Type":"ContainerStarted","Data":"6eeec2088d5b5b315c80f5373d95e72a32302ed4785813cf52cb66d12c7198e7"} Oct 30 18:38:47.256518 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:47.256462 2578 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-2s5rm" podStartSLOduration=66.460572726 podStartE2EDuration="1m8.256443961s" totalImagesPullingTime="1.795871235s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:37:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:47.255920408 +0000 UTC m=+68.969521722" watchObservedRunningTime="2026-10-30 18:38:47.256443961 +0000 UTC m=+68.970045276" Oct 30 18:38:48.957493 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:48.957450 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:49.175568 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:49.175543 2578 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:49.435338 ip-10-0-132-122 kubenswrapper[2578]: I1030 18:38:49.435224 2578 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:38:49.435398 ip-10-0-132-122 systemd[1]: Stopping Kubernetes Kubelet... Oct 30 18:38:49.453946 ip-10-0-132-122 systemd[1]: kubelet.service: Deactivated successfully. Oct 30 18:38:49.454119 ip-10-0-132-122 systemd[1]: Stopped Kubernetes Kubelet. Oct 30 18:38:49.454169 ip-10-0-132-122 systemd[1]: kubelet.service: Consumed 4.553s CPU time. Oct 30 18:38:49.464218 ip-10-0-132-122 systemd[1]: Starting Kubernetes Kubelet... Oct 30 18:38:49.559342 ip-10-0-132-122 kubenswrapper[5894]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:38:49.559342 ip-10-0-132-122 kubenswrapper[5894]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Oct 30 18:38:49.559342 ip-10-0-132-122 kubenswrapper[5894]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:38:49.559342 ip-10-0-132-122 kubenswrapper[5894]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Oct 30 18:38:49.561463 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561388 5894 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:38:49.561463 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561464 5894 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:38:49.561532 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561468 5894 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:38:49.561532 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561472 5894 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:38:49.561532 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561475 5894 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:38:49.561532 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561478 5894 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:38:49.561532 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561481 5894 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:38:49.561532 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561485 5894 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:38:49.561532 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561490 5894 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:38:49.561532 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561494 5894 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:38:49.561532 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561496 5894 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:38:49.561532 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561499 5894 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:38:49.561532 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561502 5894 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:38:49.561532 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561505 5894 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:38:49.561532 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561507 5894 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:38:49.561532 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561510 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:38:49.561532 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561513 5894 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:38:49.561532 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561515 5894 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:38:49.561532 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561518 5894 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:38:49.561532 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561521 5894 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:38:49.561532 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561523 5894 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561526 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561528 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561532 5894 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561535 5894 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561538 5894 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561541 5894 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561547 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561550 5894 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561553 5894 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561555 5894 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561558 5894 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561560 5894 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561563 5894 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561565 5894 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561568 5894 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561583 5894 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561586 5894 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561589 5894 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561592 5894 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:38:49.562011 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561594 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561597 5894 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561600 5894 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561602 5894 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561605 5894 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561608 5894 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561612 5894 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561614 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561617 5894 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561620 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561623 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561626 5894 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561628 5894 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561631 5894 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561634 5894 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561637 5894 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561639 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561642 5894 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561645 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561647 5894 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:38:49.562500 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561654 5894 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561657 5894 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561660 5894 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561662 5894 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561665 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561667 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561670 5894 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561673 5894 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561676 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561678 5894 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561681 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561683 5894 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561686 5894 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561688 5894 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561691 5894 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561694 5894 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561697 5894 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561699 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561702 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561705 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:38:49.563030 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561707 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:38:49.563508 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561710 5894 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:38:49.563508 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561713 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:38:49.563508 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561715 5894 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:38:49.563508 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561718 5894 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:38:49.563508 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561720 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:38:49.563508 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561723 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:38:49.563508 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561726 5894 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:38:49.563508 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561728 5894 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:38:49.563508 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561731 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:38:49.563508 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561734 5894 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:38:49.563508 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561736 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:38:49.563508 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561740 5894 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:38:49.563508 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561744 5894 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:38:49.563508 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561747 5894 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:38:49.563508 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561750 5894 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:38:49.563508 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561753 5894 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:38:49.563508 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561755 5894 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:38:49.563508 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561758 5894 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:38:49.563508 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561763 5894 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561767 5894 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561770 5894 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561773 5894 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561776 5894 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561778 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.561781 5894 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562589 5894 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562598 5894 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562601 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562604 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562607 5894 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562610 5894 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562613 5894 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562616 5894 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562618 5894 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562621 5894 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562623 5894 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562626 5894 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562629 5894 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:38:49.564015 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562632 5894 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:38:49.564496 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562634 5894 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:38:49.564496 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562637 5894 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:38:49.564496 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562639 5894 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:38:49.564496 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562642 5894 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:38:49.564496 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562647 5894 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:38:49.564496 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562649 5894 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:38:49.564496 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562661 5894 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:38:49.564496 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562665 5894 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:38:49.564496 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562668 5894 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:38:49.564496 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562671 5894 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:38:49.564496 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562674 5894 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:38:49.564496 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562676 5894 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:38:49.564496 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562679 5894 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:38:49.564496 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562682 5894 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:38:49.564496 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562684 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:38:49.564496 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562687 5894 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:38:49.564496 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562690 5894 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:38:49.564496 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562692 5894 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:38:49.564496 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562695 5894 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:38:49.565016 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562697 5894 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:38:49.565016 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562700 5894 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:38:49.565016 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562703 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:38:49.565016 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562705 5894 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:38:49.565016 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562707 5894 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:38:49.565016 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562710 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:38:49.565016 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562712 5894 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:38:49.565016 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562715 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:38:49.565016 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562717 5894 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:38:49.565016 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562721 5894 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:38:49.565016 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562723 5894 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:38:49.565016 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562726 5894 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:38:49.565016 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562728 5894 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:38:49.565016 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562732 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:38:49.565016 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562734 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:38:49.565016 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562737 5894 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:38:49.565016 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562739 5894 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:38:49.565016 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562744 5894 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:38:49.565016 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562747 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:38:49.565769 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562750 5894 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:38:49.565769 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562753 5894 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:38:49.565769 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562756 5894 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:38:49.565769 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562759 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:38:49.565769 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562761 5894 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:38:49.565769 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562764 5894 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:38:49.565769 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562766 5894 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:38:49.565769 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562769 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:38:49.565769 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562771 5894 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:38:49.565769 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562774 5894 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:38:49.565769 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562777 5894 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:38:49.565769 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562779 5894 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:38:49.565769 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562782 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:38:49.565769 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562784 5894 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:38:49.565769 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562787 5894 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:38:49.565769 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562789 5894 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:38:49.565769 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562792 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:38:49.565769 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562795 5894 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:38:49.565769 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562797 5894 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562800 5894 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562802 5894 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562805 5894 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562808 5894 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562811 5894 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562813 5894 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562816 5894 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562819 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562821 5894 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562824 5894 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562827 5894 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562831 5894 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562833 5894 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562836 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562840 5894 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562844 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562847 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562850 5894 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562854 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:38:49.566234 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562857 5894 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:38:49.566750 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562860 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:38:49.566750 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562863 5894 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:38:49.566750 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562865 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:38:49.566750 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562868 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:38:49.566750 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562871 5894 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:38:49.566750 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562873 5894 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:38:49.566750 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562875 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:38:49.566750 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562878 5894 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:38:49.566750 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562880 5894 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:38:49.566750 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562883 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:38:49.566750 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562885 5894 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:38:49.566750 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562888 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:38:49.566750 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562890 5894 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:38:49.566750 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562893 5894 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:38:49.566750 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.562895 5894 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:38:49.566750 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563408 5894 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:38:49.566750 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563413 5894 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:38:49.566750 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563415 5894 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:38:49.566750 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563418 5894 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:38:49.567206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563421 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:38:49.567206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563424 5894 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:38:49.567206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563426 5894 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:38:49.567206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563429 5894 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:38:49.567206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563434 5894 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:38:49.567206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563437 5894 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:38:49.567206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563439 5894 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:38:49.567206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563442 5894 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:38:49.567206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563445 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:38:49.567206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563448 5894 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:38:49.567206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563451 5894 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:38:49.567206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563453 5894 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:38:49.567206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563456 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:38:49.567206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563459 5894 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:38:49.567206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563461 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:38:49.567206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563464 5894 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:38:49.567206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563467 5894 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:38:49.567206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563469 5894 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:38:49.567206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563487 5894 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:38:49.567742 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563493 5894 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:38:49.567742 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563497 5894 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:38:49.567742 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563500 5894 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:38:49.567742 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563503 5894 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:38:49.567742 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563507 5894 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:38:49.567742 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563511 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:38:49.567742 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563515 5894 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:38:49.567742 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563518 5894 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:38:49.567742 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563521 5894 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:38:49.567742 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563524 5894 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:38:49.567742 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563527 5894 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:38:49.567742 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563529 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:38:49.567742 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563532 5894 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:38:49.567742 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563534 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:38:49.567742 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563537 5894 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:38:49.567742 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563539 5894 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:38:49.567742 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563542 5894 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:38:49.567742 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563545 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:38:49.567742 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563548 5894 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:38:49.568206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563550 5894 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:38:49.568206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563553 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:38:49.568206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563556 5894 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:38:49.568206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563562 5894 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:38:49.568206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563564 5894 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:38:49.568206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563567 5894 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:38:49.568206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563583 5894 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:38:49.568206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563586 5894 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:38:49.568206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563589 5894 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:38:49.568206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563591 5894 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:38:49.568206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563594 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:38:49.568206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563597 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:38:49.568206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563599 5894 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:38:49.568206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563603 5894 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:38:49.568206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563606 5894 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:38:49.568206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563609 5894 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:38:49.568206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563611 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:38:49.568206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563614 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:38:49.568206 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563617 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563619 5894 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563622 5894 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563626 5894 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563629 5894 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563633 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563636 5894 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563639 5894 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563642 5894 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563645 5894 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563648 5894 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563651 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563654 5894 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563657 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563660 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563662 5894 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563665 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563667 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563671 5894 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563673 5894 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:38:49.568717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563676 5894 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563678 5894 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563681 5894 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563684 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563686 5894 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563689 5894 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563691 5894 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563695 5894 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563698 5894 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563701 5894 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563703 5894 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563706 5894 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563708 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563711 5894 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563714 5894 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563717 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563719 5894 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563722 5894 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563724 5894 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563727 5894 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:38:49.569219 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563729 5894 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563733 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563735 5894 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563738 5894 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563741 5894 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.563743 5894 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563829 5894 flags.go:64] FLAG: --address="0.0.0.0" Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563836 5894 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563848 5894 flags.go:64] FLAG: --anonymous-auth="true" Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563852 5894 flags.go:64] FLAG: --application-metrics-count-limit="100" Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563857 5894 flags.go:64] FLAG: --authentication-token-webhook="false" Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563866 5894 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563870 5894 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563874 5894 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563878 5894 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563881 5894 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563884 5894 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563887 5894 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563891 5894 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563895 5894 flags.go:64] FLAG: --cgroup-root="" Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563898 5894 flags.go:64] FLAG: --cgroups-per-qos="true" Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563901 5894 flags.go:64] FLAG: --client-ca-file="" Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563904 5894 flags.go:64] FLAG: --cloud-provider="external" Oct 30 18:38:49.570037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563907 5894 flags.go:64] FLAG: --cluster-dns="[]" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563912 5894 flags.go:64] FLAG: --cluster-domain="" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563916 5894 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563919 5894 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563922 5894 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563925 5894 flags.go:64] FLAG: --container-log-max-files="5" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563929 5894 flags.go:64] FLAG: --container-log-max-size="10Mi" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563932 5894 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563936 5894 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563939 5894 flags.go:64] FLAG: --containerd-namespace="k8s.io" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563942 5894 flags.go:64] FLAG: --contention-profiling="false" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563945 5894 flags.go:64] FLAG: --cpu-cfs-quota="true" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563948 5894 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563951 5894 flags.go:64] FLAG: --cpu-manager-policy="none" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563954 5894 flags.go:64] FLAG: --cpu-manager-policy-options="" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563958 5894 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563961 5894 flags.go:64] FLAG: --enable-controller-attach-detach="true" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563964 5894 flags.go:64] FLAG: --enable-debugging-handlers="true" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563969 5894 flags.go:64] FLAG: --enable-load-reader="false" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563972 5894 flags.go:64] FLAG: --enable-server="true" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563975 5894 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563985 5894 flags.go:64] FLAG: --event-burst="100" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563988 5894 flags.go:64] FLAG: --event-qps="50" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563991 5894 flags.go:64] FLAG: --event-storage-age-limit="default=0" Oct 30 18:38:49.570656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563994 5894 flags.go:64] FLAG: --event-storage-event-limit="default=0" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.563997 5894 flags.go:64] FLAG: --eviction-hard="" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564002 5894 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564005 5894 flags.go:64] FLAG: --eviction-minimum-reclaim="" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564010 5894 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564013 5894 flags.go:64] FLAG: --eviction-soft="" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564016 5894 flags.go:64] FLAG: --eviction-soft-grace-period="" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564019 5894 flags.go:64] FLAG: --exit-on-lock-contention="false" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564023 5894 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564026 5894 flags.go:64] FLAG: --experimental-mounter-path="" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564029 5894 flags.go:64] FLAG: --fail-cgroupv1="true" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564032 5894 flags.go:64] FLAG: --fail-swap-on="true" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564035 5894 flags.go:64] FLAG: --feature-gates="" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564039 5894 flags.go:64] FLAG: --file-check-frequency="20s" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564042 5894 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564045 5894 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564048 5894 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564052 5894 flags.go:64] FLAG: --healthz-port="10248" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564055 5894 flags.go:64] FLAG: --help="false" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564058 5894 flags.go:64] FLAG: --hostname-override="ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564061 5894 flags.go:64] FLAG: --housekeeping-interval="10s" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564064 5894 flags.go:64] FLAG: --http-check-frequency="20s" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564067 5894 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Oct 30 18:38:49.571243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564071 5894 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564074 5894 flags.go:64] FLAG: --image-gc-high-threshold="85" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564077 5894 flags.go:64] FLAG: --image-gc-low-threshold="80" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564082 5894 flags.go:64] FLAG: --image-service-endpoint="" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564085 5894 flags.go:64] FLAG: --kernel-memcg-notification="false" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564087 5894 flags.go:64] FLAG: --kube-api-burst="100" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564090 5894 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564097 5894 flags.go:64] FLAG: --kube-api-qps="50" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564100 5894 flags.go:64] FLAG: --kube-reserved="" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564103 5894 flags.go:64] FLAG: --kube-reserved-cgroup="" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564107 5894 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564110 5894 flags.go:64] FLAG: --kubelet-cgroups="" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564113 5894 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564117 5894 flags.go:64] FLAG: --lock-file="" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564120 5894 flags.go:64] FLAG: --log-cadvisor-usage="false" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564123 5894 flags.go:64] FLAG: --log-flush-frequency="5s" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564126 5894 flags.go:64] FLAG: --log-json-info-buffer-size="0" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564131 5894 flags.go:64] FLAG: --log-json-split-stream="false" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564134 5894 flags.go:64] FLAG: --log-text-info-buffer-size="0" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564137 5894 flags.go:64] FLAG: --log-text-split-stream="false" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564140 5894 flags.go:64] FLAG: --logging-format="text" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564143 5894 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564147 5894 flags.go:64] FLAG: --make-iptables-util-chains="true" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564150 5894 flags.go:64] FLAG: --manifest-url="" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564153 5894 flags.go:64] FLAG: --manifest-url-header="" Oct 30 18:38:49.571828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564157 5894 flags.go:64] FLAG: --max-housekeeping-interval="15s" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564160 5894 flags.go:64] FLAG: --max-open-files="1000000" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564164 5894 flags.go:64] FLAG: --max-pods="110" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564167 5894 flags.go:64] FLAG: --maximum-dead-containers="-1" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564170 5894 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564173 5894 flags.go:64] FLAG: --memory-manager-policy="None" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564176 5894 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564179 5894 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564182 5894 flags.go:64] FLAG: --node-ip="0.0.0.0" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564185 5894 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564193 5894 flags.go:64] FLAG: --node-status-max-images="50" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564196 5894 flags.go:64] FLAG: --node-status-update-frequency="10s" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564199 5894 flags.go:64] FLAG: --oom-score-adj="-999" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564202 5894 flags.go:64] FLAG: --pod-cidr="" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564205 5894 flags.go:64] FLAG: --pod-manifest-path="" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564208 5894 flags.go:64] FLAG: --pod-max-pids="-1" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564211 5894 flags.go:64] FLAG: --pods-per-core="0" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564214 5894 flags.go:64] FLAG: --port="10250" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564217 5894 flags.go:64] FLAG: --protect-kernel-defaults="false" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564220 5894 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-039c428ae86592b74" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564224 5894 flags.go:64] FLAG: --qos-reserved="" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564228 5894 flags.go:64] FLAG: --read-only-port="10255" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564231 5894 flags.go:64] FLAG: --register-node="true" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564234 5894 flags.go:64] FLAG: --register-with-taints="" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564238 5894 flags.go:64] FLAG: --registry-burst="10" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564241 5894 flags.go:64] FLAG: --registry-qps="5" Oct 30 18:38:49.572431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564244 5894 flags.go:64] FLAG: --reserved-cpus="" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564247 5894 flags.go:64] FLAG: --reserved-memory="" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564251 5894 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564254 5894 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564258 5894 flags.go:64] FLAG: --rotate-certificates="false" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564261 5894 flags.go:64] FLAG: --rotate-server-certificates="false" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564263 5894 flags.go:64] FLAG: --runonce="false" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564266 5894 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564269 5894 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564273 5894 flags.go:64] FLAG: --seccomp-default="false" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564275 5894 flags.go:64] FLAG: --serialize-image-pulls="true" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564278 5894 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564281 5894 flags.go:64] FLAG: --storage-driver-db="cadvisor" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564284 5894 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564288 5894 flags.go:64] FLAG: --storage-driver-password="root" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564291 5894 flags.go:64] FLAG: --storage-driver-secure="false" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564295 5894 flags.go:64] FLAG: --storage-driver-table="stats" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564298 5894 flags.go:64] FLAG: --storage-driver-user="root" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564301 5894 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564304 5894 flags.go:64] FLAG: --sync-frequency="1m0s" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564307 5894 flags.go:64] FLAG: --system-cgroups="" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564310 5894 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564316 5894 flags.go:64] FLAG: --system-reserved-cgroup="" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564319 5894 flags.go:64] FLAG: --tls-cert-file="" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564322 5894 flags.go:64] FLAG: --tls-cipher-suites="[]" Oct 30 18:38:49.573073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564326 5894 flags.go:64] FLAG: --tls-min-version="" Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564330 5894 flags.go:64] FLAG: --tls-private-key-file="" Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564333 5894 flags.go:64] FLAG: --topology-manager-policy="none" Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564336 5894 flags.go:64] FLAG: --topology-manager-policy-options="" Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564339 5894 flags.go:64] FLAG: --topology-manager-scope="container" Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564342 5894 flags.go:64] FLAG: --v="2" Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564347 5894 flags.go:64] FLAG: --version="false" Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564351 5894 flags.go:64] FLAG: --vmodule="" Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564354 5894 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564358 5894 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564479 5894 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564483 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564485 5894 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564488 5894 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564491 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564494 5894 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564496 5894 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564499 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564502 5894 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564505 5894 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564508 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564510 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:38:49.573717 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564513 5894 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564516 5894 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564520 5894 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564522 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564525 5894 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564527 5894 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564530 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564532 5894 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564536 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564538 5894 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564541 5894 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564544 5894 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564547 5894 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564550 5894 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564553 5894 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564555 5894 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564558 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564561 5894 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564563 5894 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564566 5894 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:38:49.574251 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564569 5894 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564585 5894 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564589 5894 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564593 5894 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564596 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564598 5894 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564601 5894 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564604 5894 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564606 5894 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564609 5894 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564611 5894 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564614 5894 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564617 5894 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564621 5894 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564624 5894 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564627 5894 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564629 5894 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564632 5894 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564634 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564637 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:38:49.574845 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564640 5894 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564643 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564646 5894 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564650 5894 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564653 5894 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564656 5894 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564658 5894 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564661 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564664 5894 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564666 5894 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564669 5894 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564671 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564674 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564676 5894 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564679 5894 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564681 5894 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564684 5894 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564687 5894 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564689 5894 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564692 5894 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:38:49.575480 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564695 5894 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:38:49.576187 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564697 5894 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:38:49.576187 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564700 5894 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:38:49.576187 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564702 5894 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:38:49.576187 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564705 5894 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:38:49.576187 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564708 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:38:49.576187 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564711 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:38:49.576187 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564714 5894 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:38:49.576187 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564716 5894 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:38:49.576187 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564719 5894 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:38:49.576187 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564722 5894 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:38:49.576187 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564724 5894 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:38:49.576187 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564728 5894 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:38:49.576187 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564732 5894 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:38:49.576187 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564735 5894 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:38:49.576187 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564739 5894 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:38:49.576187 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564742 5894 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:38:49.576187 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564745 5894 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:38:49.576187 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564748 5894 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:38:49.576187 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564750 5894 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:38:49.577250 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564753 5894 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:38:49.577250 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564755 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:38:49.577250 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564758 5894 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:38:49.577250 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564760 5894 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:38:49.577250 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564763 5894 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:38:49.577250 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564766 5894 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:38:49.577250 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564768 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:38:49.577250 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564771 5894 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:38:49.577250 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564773 5894 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:38:49.577250 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564776 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:38:49.577250 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564778 5894 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:38:49.577250 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564781 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:38:49.577250 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564784 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:38:49.577250 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564786 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:38:49.577250 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.564811 5894 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:38:49.577250 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564845 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:38:49.577823 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564849 5894 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:38:49.577823 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564852 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:38:49.577823 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564857 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:38:49.577823 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564860 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:38:49.577823 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564863 5894 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:38:49.577823 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564866 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:38:49.577823 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564869 5894 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:38:49.577823 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564871 5894 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:38:49.577823 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564874 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:38:49.577823 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564876 5894 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:38:49.577823 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564879 5894 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:38:49.577823 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564883 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:38:49.577823 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564886 5894 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:38:49.577823 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564889 5894 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:38:49.577823 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564892 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:38:49.577823 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564894 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:38:49.577823 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564897 5894 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:38:49.577823 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564900 5894 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:38:49.577823 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564903 5894 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:38:49.578402 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564905 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:38:49.578402 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564909 5894 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:38:49.578402 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564911 5894 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:38:49.578402 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564914 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:38:49.578402 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564917 5894 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:38:49.578402 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564919 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:38:49.578402 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564922 5894 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:38:49.578402 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564924 5894 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:38:49.578402 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564927 5894 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:38:49.578402 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564930 5894 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:38:49.578402 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564932 5894 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:38:49.578402 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564935 5894 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:38:49.578402 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564938 5894 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:38:49.578402 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564940 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:38:49.578402 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564943 5894 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:38:49.578402 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564947 5894 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:38:49.578402 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564950 5894 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:38:49.578402 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564953 5894 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:38:49.578402 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564957 5894 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564961 5894 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564965 5894 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564968 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564970 5894 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564973 5894 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564977 5894 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564980 5894 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564982 5894 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564985 5894 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564988 5894 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564990 5894 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564993 5894 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564995 5894 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.564998 5894 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565000 5894 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565003 5894 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565005 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565008 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565011 5894 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:38:49.579033 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565013 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565017 5894 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565020 5894 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565022 5894 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565026 5894 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565029 5894 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565031 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565034 5894 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565037 5894 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565043 5894 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565046 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565048 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565051 5894 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565053 5894 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565056 5894 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565059 5894 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565061 5894 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565064 5894 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565067 5894 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565070 5894 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:38:49.579514 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565073 5894 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:38:49.580024 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565075 5894 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:38:49.580024 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565078 5894 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:38:49.580024 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565080 5894 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:38:49.580024 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565083 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:38:49.580024 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565086 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:38:49.580024 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565088 5894 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:38:49.580024 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565091 5894 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:38:49.580024 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565093 5894 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:38:49.580024 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565096 5894 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:38:49.580024 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565099 5894 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:38:49.580024 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565102 5894 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:38:49.580024 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565106 5894 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:38:49.580024 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565110 5894 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:38:49.580024 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565112 5894 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:38:49.580024 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565116 5894 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:38:49.580024 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565118 5894 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:38:49.580024 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565121 5894 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:38:49.580024 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565124 5894 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:38:49.580024 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565127 5894 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:38:49.580523 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565129 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:38:49.580523 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565133 5894 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:38:49.580523 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565136 5894 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:38:49.580523 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565140 5894 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:38:49.580523 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565142 5894 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:38:49.580523 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565145 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:38:49.580523 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565148 5894 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:38:49.580523 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.565150 5894 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:38:49.580523 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.569829 5894 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Oct 30 18:38:49.580523 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.569849 5894 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Oct 30 18:38:49.580523 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.569872 5894 watchdog_linux.go:95] "Systemd watchdog is not enabled" Oct 30 18:38:49.580523 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.569879 5894 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Oct 30 18:38:49.580523 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570256 5894 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:38:49.580523 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570274 5894 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:38:49.580523 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570279 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:38:49.580523 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570286 5894 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:38:49.580523 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570295 5894 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:38:49.580523 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570301 5894 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:38:49.580523 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570306 5894 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570311 5894 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570317 5894 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570321 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570326 5894 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570331 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570335 5894 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570339 5894 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570343 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570347 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570352 5894 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570356 5894 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570365 5894 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570371 5894 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570377 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570382 5894 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570393 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570398 5894 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570402 5894 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570406 5894 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:38:49.581028 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570411 5894 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:38:49.581517 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570415 5894 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:38:49.581517 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570419 5894 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:38:49.581517 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570423 5894 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:38:49.581517 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570428 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:38:49.581517 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570432 5894 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:38:49.581517 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570436 5894 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:38:49.581517 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570440 5894 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:38:49.581517 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570445 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:38:49.581517 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570449 5894 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:38:49.581517 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570453 5894 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:38:49.581517 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570458 5894 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:38:49.581517 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570462 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:38:49.581517 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570467 5894 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:38:49.581517 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570471 5894 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:38:49.581517 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570475 5894 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:38:49.581517 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570479 5894 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:38:49.581517 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570481 5894 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:38:49.581517 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570484 5894 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:38:49.581517 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570487 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570490 5894 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570494 5894 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570496 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570499 5894 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570502 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570505 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570509 5894 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570511 5894 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570514 5894 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570516 5894 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570519 5894 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570522 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570524 5894 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570527 5894 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570530 5894 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570532 5894 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570535 5894 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570537 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570540 5894 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:38:49.581994 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570542 5894 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:38:49.582562 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570545 5894 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:38:49.582562 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570547 5894 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:38:49.582562 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570550 5894 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:38:49.582562 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570553 5894 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:38:49.582562 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570555 5894 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:38:49.582562 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570558 5894 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:38:49.582562 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570561 5894 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:38:49.582562 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570563 5894 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:38:49.582562 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570565 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:38:49.582562 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570569 5894 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:38:49.582562 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570590 5894 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:38:49.582562 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570595 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:38:49.582562 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570598 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:38:49.582562 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570607 5894 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:38:49.582562 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570611 5894 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:38:49.582562 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570613 5894 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:38:49.582562 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570616 5894 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:38:49.582562 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570619 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:38:49.582562 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570622 5894 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570624 5894 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570627 5894 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570629 5894 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570632 5894 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570634 5894 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570637 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570639 5894 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570642 5894 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570645 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570647 5894 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570650 5894 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570652 5894 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570655 5894 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570658 5894 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570661 5894 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570664 5894 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570667 5894 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570670 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570673 5894 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:38:49.583054 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570675 5894 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:38:49.583621 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570678 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:38:49.583621 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.570709 5894 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:38:49.583621 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570853 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:38:49.583621 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570858 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:38:49.583621 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570861 5894 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:38:49.583621 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570864 5894 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:38:49.583621 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570867 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:38:49.583621 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570871 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:38:49.583621 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570874 5894 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:38:49.583621 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570877 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:38:49.583621 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570881 5894 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:38:49.583621 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570885 5894 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:38:49.583621 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570888 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:38:49.583621 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570890 5894 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:38:49.583621 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570893 5894 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570896 5894 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570898 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570901 5894 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570904 5894 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570906 5894 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570909 5894 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570911 5894 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570914 5894 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570917 5894 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570920 5894 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570922 5894 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570925 5894 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570927 5894 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570930 5894 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570932 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570935 5894 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570937 5894 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570940 5894 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570942 5894 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:38:49.583993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570945 5894 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:38:49.584474 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570948 5894 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:38:49.584474 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570950 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:38:49.584474 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570953 5894 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:38:49.584474 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570955 5894 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:38:49.584474 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570958 5894 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:38:49.584474 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570962 5894 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:38:49.584474 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570966 5894 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:38:49.584474 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570969 5894 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:38:49.584474 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570972 5894 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:38:49.584474 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570975 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:38:49.584474 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570978 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:38:49.584474 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570980 5894 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:38:49.584474 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570983 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:38:49.584474 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570985 5894 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:38:49.584474 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570988 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:38:49.584474 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570991 5894 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:38:49.584474 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570993 5894 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:38:49.584474 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570996 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:38:49.584474 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.570998 5894 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571001 5894 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571003 5894 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571006 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571008 5894 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571011 5894 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571013 5894 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571016 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571018 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571021 5894 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571023 5894 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571026 5894 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571028 5894 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571031 5894 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571033 5894 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571036 5894 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571038 5894 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571041 5894 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571044 5894 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571046 5894 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:38:49.584984 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571049 5894 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571052 5894 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571055 5894 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571057 5894 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571060 5894 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571062 5894 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571065 5894 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571067 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571070 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571072 5894 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571075 5894 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571077 5894 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571080 5894 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571083 5894 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571085 5894 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571088 5894 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571090 5894 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571093 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571095 5894 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571098 5894 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:38:49.585465 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571100 5894 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:38:49.585964 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571103 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:38:49.585964 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571105 5894 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:38:49.585964 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571108 5894 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:38:49.585964 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571111 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:38:49.585964 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571113 5894 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:38:49.585964 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571116 5894 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:38:49.585964 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571118 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:38:49.585964 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571121 5894 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:38:49.585964 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571124 5894 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:38:49.585964 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571127 5894 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:38:49.585964 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571130 5894 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:38:49.585964 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571132 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:38:49.585964 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571135 5894 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:38:49.585964 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571138 5894 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:38:49.585964 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.571161 5894 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Oct 30 18:38:49.585964 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571192 5894 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Oct 30 18:38:49.586530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571196 5894 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Oct 30 18:38:49.586530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571199 5894 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Oct 30 18:38:49.586530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571202 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Oct 30 18:38:49.586530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571205 5894 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Oct 30 18:38:49.586530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571208 5894 feature_gate.go:386] unrecognized feature gate: TLSAdherence Oct 30 18:38:49.586530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571210 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Oct 30 18:38:49.586530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571213 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Oct 30 18:38:49.586530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571215 5894 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Oct 30 18:38:49.586530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571218 5894 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Oct 30 18:38:49.586530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571220 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Oct 30 18:38:49.586530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571223 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Oct 30 18:38:49.586530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571225 5894 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Oct 30 18:38:49.586530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571228 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Oct 30 18:38:49.586530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571232 5894 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Oct 30 18:38:49.586530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571238 5894 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Oct 30 18:38:49.586530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571241 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Oct 30 18:38:49.586530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571243 5894 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Oct 30 18:38:49.586530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571247 5894 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Oct 30 18:38:49.586530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571250 5894 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571253 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571256 5894 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571258 5894 feature_gate.go:386] unrecognized feature gate: Example2 Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571261 5894 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571264 5894 feature_gate.go:386] unrecognized feature gate: EventTTL Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571267 5894 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571269 5894 feature_gate.go:386] unrecognized feature gate: OSStreams Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571272 5894 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571274 5894 feature_gate.go:386] unrecognized feature gate: NewOLM Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571277 5894 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571280 5894 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571283 5894 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571286 5894 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571288 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571291 5894 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571293 5894 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571296 5894 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571298 5894 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571301 5894 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Oct 30 18:38:49.587304 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571303 5894 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Oct 30 18:38:49.588284 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571306 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Oct 30 18:38:49.588284 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571308 5894 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Oct 30 18:38:49.588284 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571311 5894 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Oct 30 18:38:49.588284 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571313 5894 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Oct 30 18:38:49.588284 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571317 5894 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Oct 30 18:38:49.588284 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571320 5894 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Oct 30 18:38:49.588284 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571323 5894 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Oct 30 18:38:49.588284 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571328 5894 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Oct 30 18:38:49.588284 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571331 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Oct 30 18:38:49.588284 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571334 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Oct 30 18:38:49.588284 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571337 5894 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Oct 30 18:38:49.588284 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571340 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Oct 30 18:38:49.588284 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571343 5894 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Oct 30 18:38:49.588284 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571346 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Oct 30 18:38:49.588284 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571349 5894 feature_gate.go:386] unrecognized feature gate: InsightsConfig Oct 30 18:38:49.588284 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571352 5894 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Oct 30 18:38:49.588284 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571355 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Oct 30 18:38:49.588284 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571357 5894 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Oct 30 18:38:49.588284 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571360 5894 feature_gate.go:386] unrecognized feature gate: SignatureStores Oct 30 18:38:49.589173 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571362 5894 feature_gate.go:386] unrecognized feature gate: DualReplica Oct 30 18:38:49.589173 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571365 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Oct 30 18:38:49.589173 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571368 5894 feature_gate.go:386] unrecognized feature gate: KMSEncryption Oct 30 18:38:49.589173 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571370 5894 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Oct 30 18:38:49.589173 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571373 5894 feature_gate.go:386] unrecognized feature gate: NetworkConnect Oct 30 18:38:49.589173 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571376 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Oct 30 18:38:49.589173 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571378 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Oct 30 18:38:49.589173 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571381 5894 feature_gate.go:386] unrecognized feature gate: OVNObservability Oct 30 18:38:49.589173 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571383 5894 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Oct 30 18:38:49.589173 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571385 5894 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Oct 30 18:38:49.589173 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571388 5894 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Oct 30 18:38:49.589173 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571390 5894 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Oct 30 18:38:49.589173 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571393 5894 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Oct 30 18:38:49.589173 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571396 5894 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Oct 30 18:38:49.589173 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571398 5894 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Oct 30 18:38:49.589173 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571401 5894 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Oct 30 18:38:49.589173 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571404 5894 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Oct 30 18:38:49.589173 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571407 5894 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Oct 30 18:38:49.589173 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571409 5894 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571411 5894 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571416 5894 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571421 5894 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571424 5894 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571426 5894 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571429 5894 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571431 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571434 5894 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571436 5894 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571439 5894 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571442 5894 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571444 5894 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571447 5894 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571449 5894 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571452 5894 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571455 5894 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571457 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571460 5894 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571463 5894 feature_gate.go:386] unrecognized feature gate: EVPN Oct 30 18:38:49.590077 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571465 5894 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Oct 30 18:38:49.591055 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571468 5894 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Oct 30 18:38:49.591055 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571471 5894 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Oct 30 18:38:49.591055 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571473 5894 feature_gate.go:386] unrecognized feature gate: Example Oct 30 18:38:49.591055 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571476 5894 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Oct 30 18:38:49.591055 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571479 5894 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Oct 30 18:38:49.591055 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571481 5894 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Oct 30 18:38:49.591055 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571484 5894 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Oct 30 18:38:49.591055 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:49.571486 5894 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Oct 30 18:38:49.591055 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.571537 5894 server.go:957] "Client rotation is on, will bootstrap in background" Oct 30 18:38:49.591055 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.572556 5894 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Oct 30 18:38:49.591055 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.572634 5894 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Oct 30 18:38:49.591055 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.572817 5894 server.go:1014] "Starting client certificate rotation" Oct 30 18:38:49.591055 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.572946 5894 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Oct 30 18:38:49.591055 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.572996 5894 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-10-29 18:32:38 +0000 UTC" deadline="2028-04-30 08:20:22.528357818 +0000 UTC" Oct 30 18:38:49.591055 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.573022 5894 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13141h41m32.955337054s" Oct 30 18:38:49.591055 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.573289 5894 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:38:49.592030 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.574308 5894 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Oct 30 18:38:49.592030 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.575370 5894 log.go:25] "Validated CRI v1 runtime API" Oct 30 18:38:49.592030 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.577772 5894 log.go:25] "Validated CRI v1 image API" Oct 30 18:38:49.592030 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.578462 5894 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Oct 30 18:38:49.592030 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.580842 5894 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 b9b99cef-be28-4f4c-997a-4810993ee868:/dev/nvme0n1p4 c588d081-9b52-4f1d-802f-6799137d242e:/dev/nvme0n1p3] Oct 30 18:38:49.592272 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.580859 5894 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/06192cb253d0f89797abd9283cd05e09470c34b4bc601a192fd03ad567b03272/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/06192cb253d0f89797abd9283cd05e09470c34b4bc601a192fd03ad567b03272/userdata/shm major:0 minor:341 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/10e2ded499f32c29040bcaac33da17b080ba2bf5682452b99e2e12d16855c1fe/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/10e2ded499f32c29040bcaac33da17b080ba2bf5682452b99e2e12d16855c1fe/userdata/shm major:0 minor:246 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/12afdaecda545f8c43bef504323a532ac5a46dbabd623069abfdac6501c84655/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/12afdaecda545f8c43bef504323a532ac5a46dbabd623069abfdac6501c84655/userdata/shm major:0 minor:440 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1d9056edc0e80f4fa89a98a908130771563b58b346ba3d202690e1907398462f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1d9056edc0e80f4fa89a98a908130771563b58b346ba3d202690e1907398462f/userdata/shm major:0 minor:380 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1def0758feaa34df841ee35ec0c9744f1652ec4e3db9d160b8212fff4fb5ed04/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1def0758feaa34df841ee35ec0c9744f1652ec4e3db9d160b8212fff4fb5ed04/userdata/shm major:0 minor:63 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/234c59b104002e47f821c4f580fd993ad867affa8bd4ad0a833ceb243b979f03/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/234c59b104002e47f821c4f580fd993ad867affa8bd4ad0a833ceb243b979f03/userdata/shm major:0 minor:82 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/28a4b9b1d84bfb15c36875674bcc12c565c22f842974116ffe42f7d4f3c326a0/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/28a4b9b1d84bfb15c36875674bcc12c565c22f842974116ffe42f7d4f3c326a0/userdata/shm major:0 minor:216 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2fd69b6a889132d82f4380a7c8a7a16109a844804e2c03a418e94af8a666f6bf/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2fd69b6a889132d82f4380a7c8a7a16109a844804e2c03a418e94af8a666f6bf/userdata/shm major:0 minor:61 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4336e440d847cb102b5a84cf5db4e4868879b7a45b75ad04742b4b0303ca407f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4336e440d847cb102b5a84cf5db4e4868879b7a45b75ad04742b4b0303ca407f/userdata/shm major:0 minor:419 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4656cde633a452376680ea5749895c5b4884b3df064fd06daafcf44e6f1ff605/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4656cde633a452376680ea5749895c5b4884b3df064fd06daafcf44e6f1ff605/userdata/shm major:0 minor:86 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/4971600d17965249e566952e3d5de9e09491c9d11640b0f63933dc353ea1ab09/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/4971600d17965249e566952e3d5de9e09491c9d11640b0f63933dc353ea1ab09/userdata/shm major:0 minor:228 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/510a7c62896ac8e28e4f9b7c8a70f158b31b85c7f39b6faae1447460f7418a12/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/510a7c62896ac8e28e4f9b7c8a70f158b31b85c7f39b6faae1447460f7418a12/userdata/shm major:0 minor:90 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/53177fd6712175b6d60e8410ffa77d7e8f380168e452c864291dfbcb52a18432/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/53177fd6712175b6d60e8410ffa77d7e8f380168e452c864291dfbcb52a18432/userdata/shm major:0 minor:287 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5dd77f7e140b9dab12799c0b35a5dc1c3df7894456b2c54abf57f224455ee922/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5dd77f7e140b9dab12799c0b35a5dc1c3df7894456b2c54abf57f224455ee922/userdata/shm major:0 minor:384 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6308f380c32bcbf4b59577d164b1513e052edcfddce9b3a9e4e0392784d389ae/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6308f380c32bcbf4b59577d164b1513e052edcfddce9b3a9e4e0392784d389ae/userdata/shm major:0 minor:233 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6898926c00a6bae4a7eaa36bbcf4186056d4fdeeb32b72ad64de694e4ff305b3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6898926c00a6bae4a7eaa36bbcf4186056d4fdeeb32b72ad64de694e4ff305b3/userdata/shm major:0 minor:462 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6a71eac430b3a11c70f0218e4d5783578dcf216b23d800a92b73ec44e02196c5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6a71eac430b3a11c70f0218e4d5783578dcf216b23d800a92b73ec44e02196c5/userdata/shm major:0 minor:214 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/70efc2349fb099a70d72f8c4332cb650c93cabd5374020a22e3c3ca0c2b662a6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/70efc2349fb099a70d72f8c4332cb650c93cabd5374020a22e3c3ca0c2b662a6/userdata/shm major:0 minor:339 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/756fad8be1817e3079ad90464555d4d70a5101b1568c397ddb5ec583a41bbe42/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/756fad8be1817e3079ad90464555d4d70a5101b1568c397ddb5ec583a41bbe42/userdata/shm major:0 minor:242 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9809304bf3e9a708db1a5b926a67a728cc1e9c0f3308b9f01a9f57dd56787f89/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9809304bf3e9a708db1a5b926a67a728cc1e9c0f3308b9f01a9f57dd56787f89/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a3d0c2abffcf4a28f3b6343dea6d745d27fd48780411b32639a996672e47416d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a3d0c2abffcf4a28f3b6343dea6d745d27fd48780411b32639a996672e47416d/userdata/shm major:0 minor:94 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/a590889638ad55563f8b4a4f6a2914ff008c4675b8cc971a553efbb73101bc21/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/a590889638ad55563f8b4a4f6a2914ff008c4675b8cc971a553efbb73101bc21/userdata/shm major:0 minor:98 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b6b8b467a9a9c17d405b5f134e0a4f52618908c014498e12875220ea4be2ae9e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b6b8b467a9a9c17d405b5f134e0a4f52618908c014498e12875220ea4be2ae9e/userdata/shm major:0 minor:244 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/b7ce03394c927b190fb2b2d52042e55bddcbb68628f97ae7b932e5e2266f68d3/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/b7ce03394c927b190fb2b2d52042e55bddcbb68628f97ae7b932e5e2266f68d3/userdata/shm major:0 minor:357 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ba4f4ec128c62434c0766741f1006e473a81b720369bc4ea6e3a8d1cadcb6c31/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ba4f4ec128c62434c0766741f1006e473a81b720369bc4ea6e3a8d1cadcb6c31/userdata/shm major:0 minor:464 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/bdda3f2b9e852331ec036d72d43573711d031903faddb053be331836e4cf3d6d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/bdda3f2b9e852331ec036d72d43573711d031903faddb053be331836e4cf3d6d/userdata/shm major:0 minor:227 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ccf8d80ee7794e49416a9b3d06822143ece1c2af772de7d1b76b0faa5bbd58d5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ccf8d80ee7794e49416a9b3d06822143ece1c2af772de7d1b76b0faa5bbd58d5/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/d11cbe37ba26264691e609ca8fc29e574c0b34e9a14cc855094fdc0350d02859/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/d11cbe37ba26264691e609ca8fc29e574c0b34e9a14cc855094fdc0350d02859/userdata/shm major:0 minor:386 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e035e6d046c09feaf39dad9de2fed20fa7f62fac403a143af5925da680b22ca2/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e035e6d046c09feaf39dad9de2fed20fa7f62fac403a143af5925da680b22ca2/userdata/shm major:0 minor:60 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e09c65370ab654a610997634985c1f985d1ca8b2f217adcfd54b7bbd0af1af48/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e09c65370ab654a610997634985c1f985d1ca8b2f217adcfd54b7bbd0af1af48/userdata/shm major:0 minor:235 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e5622c316fa0601a6308ab7fced872c54601ccf1c0f8b094ac3870a49d6fc09e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e5622c316fa0601a6308ab7fced872c54601ccf1c0f8b094ac3870a49d6fc09e/userdata/shm major:0 minor:442 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e9aa7c0d77120340e0ef3c531e83d699e403f7825a9c67d1ee67556aa802cc05/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e9aa7c0d77120340e0ef3c531e83d699e403f7825a9c67d1ee67556aa802cc05/userdata/shm major:0 minor:238 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/f7b2298ae1b4cf4ef2cbb4e235848ecc9fdfe1cbcb06169900783b1ecdec7e47/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/f7b2298ae1b4cf4ef2cbb4e235848ecc9fdfe1cbcb06169900783b1ecdec7e47/userdata/shm major:0 minor:62 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ff0ec68a90647ce926cdb4307ff2be59548cfbdc73a2e046c61eade4003464e8/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ff0ec68a90647ce926cdb4307ff2be59548cfbdc73a2e046c61eade4003464e8/userdata/shm major:0 minor:335 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/15f5b642-cd06-42ad-9832-dcfe438ea4e5/volumes/kubernetes.io~projected/kube-api-access-6tt86:{mountpoint:/var/lib/kubelet/pods/15f5b642-cd06-42ad-9832-dcfe438ea4e5/volumes/kubernetes.io~projected/kube-api-access-6tt86 major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2797391b-fd3d-4222-82c8-b28a2d16c9e3/volumes/kubernetes.io~projected/kube-api-access-lg2js:{mountpoint:/var/lib/kubelet/pods/2797391b-fd3d-4222-82c8-b28a2d16c9e3/volumes/kubernetes.io~projected/kube-api-access-lg2js major:0 minor:202 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2797391b-fd3d-4222-82c8-b28a2d16c9e3/volumes/kubernetes.io~secret/cluster-monitoring-operator-tls:{mountpoint:/var/lib/kubelet/pods/2797391b-fd3d-4222-82c8-b28a2d16c9e3/volumes/kubernetes.io~secret/cluster-monitoring-operator-tls major:0 minor:375 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/290aeefa-831a-4e2d-bc4c-11defdfee2d0/volumes/kubernetes.io~projected/kube-api-access-qwgvn:{mountpoint:/var/lib/kubelet/pods/290aeefa-831a-4e2d-bc4c-11defdfee2d0/volumes/kubernetes.io~projected/kube-api-access-qwgvn major:0 minor:325 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/2bfeb3c6-d71a-4b5e-96db-198da488f7a9/volumes/kubernetes.io~projected/kube-api-access-qndwx:{mountpoint:/var/lib/kubelet/pods/2bfeb3c6-d71a-4b5e-96db-198da488f7a9/volumes/kubernetes.io~projected/kube-api-access-qndwx major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3180fff2-f289-47e6-a37c-34c5dff50aa2/volumes/kubernetes.io~projected/kube-api-access-n4p6x:{mountpoint:/var/lib/kubelet/pods/3180fff2-f289-47e6-a37c-34c5dff50aa2/volumes/kubernetes.io~projected/kube-api-access-n4p6x major:0 minor:225 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3180fff2-f289-47e6-a37c-34c5dff50aa2/volumes/kubernetes.io~secret/default-certificate:{mountpoint:/var/lib/kubelet/pods/3180fff2-f289-47e6-a37c-34c5dff50aa2/volumes/kubernetes.io~secret/default-certificate major:0 minor:212 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3180fff2-f289-47e6-a37c-34c5dff50aa2/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/3180fff2-f289-47e6-a37c-34c5dff50aa2/volumes/kubernetes.io~secret/metrics-certs major:0 minor:383 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3180fff2-f289-47e6-a37c-34c5dff50aa2/volumes/kubernetes.io~secret/stats-auth:{mountpoint:/var/lib/kubelet/pods/3180fff2-f289-47e6-a37c-34c5dff50aa2/volumes/kubernetes.io~secret/stats-auth major:0 minor:209 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/34a1d011-b30b-49b8-bc75-bfb9dbba3817/volumes/kubernetes.io~projected/kube-api-access-4zfjf:{mountpoint:/var/lib/kubelet/pods/34a1d011-b30b-49b8-bc75-bfb9dbba3817/volumes/kubernetes.io~projected/kube-api-access-4zfjf major:0 minor:203 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3d5e33b3-69cd-4fc7-8366-14ffa038713e/volumes/kubernetes.io~projected/kube-api-access-4w8p5:{mountpoint:/var/lib/kubelet/pods/3d5e33b3-69cd-4fc7-8366-14ffa038713e/volumes/kubernetes.io~projected/kube-api-access-4w8p5 major:0 minor:330 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/3d5e33b3-69cd-4fc7-8366-14ffa038713e/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/3d5e33b3-69cd-4fc7-8366-14ffa038713e/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:414 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00/volumes/kubernetes.io~projected/kube-api-access-qjh2m:{mountpoint:/var/lib/kubelet/pods/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00/volumes/kubernetes.io~projected/kube-api-access-qjh2m major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a88dde4-2103-476d-81c4-4b255cff4828/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/4a88dde4-2103-476d-81c4-4b255cff4828/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:223 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a88dde4-2103-476d-81c4-4b255cff4828/volumes/kubernetes.io~projected/kube-api-access-v87g2:{mountpoint:/var/lib/kubelet/pods/4a88dde4-2103-476d-81c4-4b255cff4828/volumes/kubernetes.io~projected/kube-api-access-v87g2 major:0 minor:219 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a88dde4-2103-476d-81c4-4b255cff4828/volumes/kubernetes.io~projected/registry-tls:{mountpoint:/var/lib/kubelet/pods/4a88dde4-2103-476d-81c4-4b255cff4828/volumes/kubernetes.io~projected/registry-tls major:0 minor:218 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a88dde4-2103-476d-81c4-4b255cff4828/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/4a88dde4-2103-476d-81c4-4b255cff4828/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:210 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4a88dde4-2103-476d-81c4-4b255cff4828/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/4a88dde4-2103-476d-81c4-4b255cff4828/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:205 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/4b9a31e1-73c6-4617-ac72-296e03e928f2/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/4b9a31e1-73c6-4617-ac72-296e03e928f2/volumes/kubernetes.io~secret/agent-certs major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/51d9053b-abef-45c8-bd1a-67343812dcd9/volumes/kubernetes.io~projected/kube-api-access-lvrqw:{mountpoint:/var/lib/kubelet/pods/51d9053b-abef-45c8-bd1a-67343812dcd9/volumes/kubernetes.io~projected/kube-api-access-lvrqw major:0 minor:237 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/63658d25-6d63-42e2-9755-8ca66a322d61/volumes/kubernetes.io~projected/kube-api-access-7hlr2:{mountpoint:/var/lib/kubelet/pods/63658d25-6d63-42e2-9755-8ca66a322d61/volumes/kubernetes.io~projected/kube-api-access-7hlr2 major:0 minor:240 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/63658d25-6d63-42e2-9755-8ca66a322d61/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/63658d25-6d63-42e2-9755-8ca66a322d61/volumes/kubernetes.io~secret/cert major:0 minor:232 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/67b57c01-e4a3-4ae1-ab6e-9b386e181adb/volumes/kubernetes.io~projected/kube-api-access-ghbwl:{mountpoint:/var/lib/kubelet/pods/67b57c01-e4a3-4ae1-ab6e-9b386e181adb/volumes/kubernetes.io~projected/kube-api-access-ghbwl major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/67b57c01-e4a3-4ae1-ab6e-9b386e181adb/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/67b57c01-e4a3-4ae1-ab6e-9b386e181adb/volumes/kubernetes.io~secret/serving-cert major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6b091344-3cff-4f72-9512-02499eb9a9ef/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/6b091344-3cff-4f72-9512-02499eb9a9ef/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6b091344-3cff-4f72-9512-02499eb9a9ef/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/6b091344-3cff-4f72-9512-02499eb9a9ef/volumes/kubernetes.io~empty-dir/tmp major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/6b091344-3cff-4f72-9512-02499eb9a9ef/volumes/kubernetes.io~projected/kube-api-access-thmjv:{mountpoint:/var/lib/kubelet/pods/6b091344-3cff-4f72-9512-02499eb9a9ef/volumes/kubernetes.io~projected/kube-api-access-thmjv major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/74990028-c4d9-40d7-b1dc-b45b9f3ccbf2/volumes/kubernetes.io~projected/kube-api-access-xh5hg:{mountpoint:/var/lib/kubelet/pods/74990028-c4d9-40d7-b1dc-b45b9f3ccbf2/volumes/kubernetes.io~projected/kube-api-access-xh5hg major:0 minor:337 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca/volumes/kubernetes.io~projected/kube-api-access-b6f6l:{mountpoint:/var/lib/kubelet/pods/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca/volumes/kubernetes.io~projected/kube-api-access-b6f6l major:0 minor:356 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca/volumes/kubernetes.io~secret/signing-key:{mountpoint:/var/lib/kubelet/pods/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca/volumes/kubernetes.io~secret/signing-key major:0 minor:355 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7a4adea8-c5dc-4a4d-ad54-e985838fa6c5/volumes/kubernetes.io~projected/kube-api-access-w6srj:{mountpoint:/var/lib/kubelet/pods/7a4adea8-c5dc-4a4d-ad54-e985838fa6c5/volumes/kubernetes.io~projected/kube-api-access-w6srj major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ca07043-d7b5-4660-87d4-2a92bf5dacbf/volumes/kubernetes.io~projected/kube-api-access-6srz4:{mountpoint:/var/lib/kubelet/pods/7ca07043-d7b5-4660-87d4-2a92bf5dacbf/volumes/kubernetes.io~projected/kube-api-access-6srz4 major:0 minor:204 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7ca07043-d7b5-4660-87d4-2a92bf5dacbf/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/7ca07043-d7b5-4660-87d4-2a92bf5dacbf/volumes/kubernetes.io~secret/serving-cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7eed8239-b695-41f7-968f-b3a02659b44c/volumes/kubernetes.io~projected/kube-api-access-wqsv7:{mountpoint:/var/lib/kubelet/pods/7eed8239-b695-41f7-968f-b3a02659b44c/volumes/kubernetes.io~projected/kube-api-access-wqsv7 major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/89271a7f-e9c4-43dd-a320-54900b3fb8b7/volumes/kubernetes.io~projected/kube-api-access-8blzq:{mountpoint:/var/lib/kubelet/pods/89271a7f-e9c4-43dd-a320-54900b3fb8b7/volumes/kubernetes.io~projected/kube-api-access-8blzq major:0 minor:224 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/89271a7f-e9c4-43dd-a320-54900b3fb8b7/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/89271a7f-e9c4-43dd-a320-54900b3fb8b7/volumes/kubernetes.io~secret/serving-cert major:0 minor:213 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8c9b04e2-b099-4704-8786-b64a8991d0f4/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/8c9b04e2-b099-4704-8786-b64a8991d0f4/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:460 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2/volumes/kubernetes.io~projected/kube-api-access-b9hs4:{mountpoint:/var/lib/kubelet/pods/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2/volumes/kubernetes.io~projected/kube-api-access-b9hs4 major:0 minor:56 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2/volumes/kubernetes.io~secret/metrics-certs major:0 minor:461 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/91b2497b-8770-4c64-9b1f-e2ec8744ccf9/volumes/kubernetes.io~projected/kube-api-access-cbjt2:{mountpoint:/var/lib/kubelet/pods/91b2497b-8770-4c64-9b1f-e2ec8744ccf9/volumes/kubernetes.io~projected/kube-api-access-cbjt2 major:0 minor:338 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9cd97653-a745-4fd6-9a6c-71aaa742b415/volumes/kubernetes.io~projected/kube-api-access-2c2tj:{mountpoint:/var/lib/kubelet/pods/9cd97653-a745-4fd6-9a6c-71aaa742b415/volumes/kubernetes.io~projected/kube-api-access-2c2tj major:0 minor:222 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/9cd97653-a745-4fd6-9a6c-71aaa742b415/volumes/kubernetes.io~secret/samples-operator-tls:{mountpoint:/var/lib/kubelet/pods/9cd97653-a745-4fd6-9a6c-71aaa742b415/volumes/kubernetes.io~secret/samples-operator-tls major:0 minor:211 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f/volumes/kubernetes.io~projected/kube-api-access-v72l5:{mountpoint:/var/lib/kubelet/pods/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f/volumes/kubernetes.io~projected/kube-api-access-v72l5 major:0 minor:239 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f/volumes/kubernetes.io~secret/metrics-tls major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a36f2ae5-c6ca-49cf-a775-82a97e92f2c4/volumes/kubernetes.io~projected/kube-api-access-m5w86:{mountpoint:/var/lib/kubelet/pods/a36f2ae5-c6ca-49cf-a775-82a97e92f2c4/volumes/kubernetes.io~projected/kube-api-access-m5w86 major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bb52d69e-0e3f-46b5-8491-a2be5a912d29/volumes/kubernetes.io~projected/kube-api-access-2q9pq:{mountpoint:/var/lib/kubelet/pods/bb52d69e-0e3f-46b5-8491-a2be5a912d29/volumes/kubernetes.io~projected/kube-api-access-2q9pq major:0 minor:226 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c135952e-12c3-4a65-bd3d-0d0b1aba49d1/volumes/kubernetes.io~secret/networking-console-plugin-cert:{mountpoint:/var/lib/kubelet/pods/c135952e-12c3-4a65-bd3d-0d0b1aba49d1/volumes/kubernetes.io~secret/networking-console-plugin-cert major:0 minor:382 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/c89d1725-2dbb-49dc-92b1-218935ce8a74/volumes/kubernetes.io~projected/kube-api-access-fkqwc:{mountpoint:/var/lib/kubelet/pods/c89d1725-2dbb-49dc-92b1-218935ce8a74/volumes/kubernetes.io~projected/kube-api-access-fkqwc major:0 minor:52 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/cd195fdf-8022-4279-8eca-15d145d59038/volumes/kubernetes.io~projected/kube-api-access-6l7k8:{mountpoint:/var/lib/kubelet/pods/cd195fdf-8022-4279-8eca-15d145d59038/volumes/kubernetes.io~projected/kube-api-access-6l7k8 major:0 minor:268 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce6f0c54-70e4-4d45-8bc3-2823a815f0db/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/ce6f0c54-70e4-4d45-8bc3-2823a815f0db/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce6f0c54-70e4-4d45-8bc3-2823a815f0db/volumes/kubernetes.io~projected/kube-api-access-xmb9d:{mountpoint:/var/lib/kubelet/pods/ce6f0c54-70e4-4d45-8bc3-2823a815f0db/volumes/kubernetes.io~projected/kube-api-access-xmb9d major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ce6f0c54-70e4-4d45-8bc3-2823a815f0db/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/ce6f0c54-70e4-4d45-8bc3-2823a815f0db/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e2455903-bb15-41e2-8d91-51df8775b033/volumes/kubernetes.io~projected/kube-api-access-ztsmk:{mountpoint:/var/lib/kubelet/pods/e2455903-bb15-41e2-8d91-51df8775b033/volumes/kubernetes.io~projected/kube-api-access-ztsmk major:0 minor:51 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e65f9eb5-9e22-4c76-a1b5-e2250003faa6/volumes/kubernetes.io~projected/kube-api-access-k44bp:{mountpoint:/var/lib/kubelet/pods/e65f9eb5-9e22-4c76-a1b5-e2250003faa6/volumes/kubernetes.io~projected/kube-api-access-k44bp major:0 minor:221 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/e65f9eb5-9e22-4c76-a1b5-e2250003faa6/volumes/kubernetes.io~secret/serving-cert:{mountpoint:/var/lib/kubelet/pods/e65f9eb5-9e22-4c76-a1b5-e2250003faa6/volumes/kubernetes.io~secret/serving-cert major:0 minor:208 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/c17470c144b69651cbc40c06f1a18a41b7fe525c325573da81643ee12e5b214c/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/fd9552e5ab9b3e026887aa2264352b03634baa18f0df9ca84e01ef4ed772b81e/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/c768e404c708d868c980b1e95ffde12aec1a2ff1be4a40215dc2c5b5372c1ac1/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/ee4f3d1c2480114bdda23963f1f5e69002d2fbedc80eb0d9be54949b86d6feff/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-114:{mountpoint:/var/lib/containers/storage/overlay/f593c19e4139df972e74339d64a4c7c5548899f07d83f8c9bf5e5c6da36706fc/merged major:0 minor:114 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/8a14d1721f58ca7bdf7e7b5a7f157e9d55f1240996f4dde2e7b2ccb9fbcf8ae8/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/1259246c8dca68f2d51763695cee73cd1cb4850b84f7c9167fb48241e86d739d/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/a6983eb81d3d6503b35b509b48910b9bfad1adf88f9dcf6dc218209aa44c97ed/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/81abd9f6a6f740dd6dfbe73102d369e797c133077e0788d77c042f97babd3b2e/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/302640a336a9cca89141723db5792816c30f9ed50c7b74ee3362f81c450c0c2f/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/ee3959ce8bdb6e57ac4fd6f8c48e158dc7cb87ad09b35d978930ed01bec68783/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-146:{mountpoint:/var/lib/containers/storage/overlay/daaa263b9fd622d6c19b62c8cd19a335d5a25d787e5377d35cdadea5df031cd6/merged major:0 minor:146 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/acd991e12d0c993be1dd746ff3c63cd9484e4c263487ec6ccae833a7875cb863/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-154:{mountpoint:/var/lib/containers/storage/overlay/d55502b88384912d2e6f9093490ad6560e687d152d39a0a4f4ef704f6c5eb383/merged major:0 minor:154 fsType:overlay blockSize:0} overlay_0-158:{mountpoint:/var/lib/containers/storage/overlay/a4136c09fb5f7b6ed13c859ea950f183652fc226872ffbf83e9354f75f7ac40c/merged major:0 minor:158 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/ae2747225e765687f7a8f2c5a814bb67a1621e8e244e8862d9d23a2c1dc0feb8/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-172:{mountpoint:/var/lib/containers/storage/overlay/4d838b24283b0eec186a217cc6d0fac00ca54af0bd8a3550e8212f0933bb6532/merged major:0 minor:172 fsType:overlay blockSize:0} overlay_0-177:{mountpoint:/var/lib/containers/storage/overlay/27561e22a1fddef8151f311346477f71a6b83e6b208b730f5e19e51f6100b9e1/merged major:0 minor:177 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/b845fa1ba50b5aa58dd49f26db872d54c6044320508fe79e41d2fc3d97b3936d/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/e404fc0ac266aeeca8f5334db30f5956f4fd94f2273f27b2a7171d01bed56754/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/abf4d8229ef00783d9255535e190c1528d24c190921675aa1caa34cf0576fa5c/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/fbbc80e309a9c847ea850235d9953a8ef04bf5a5016f278704900e7461706803/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-250:{mountpoint:/var/lib/containers/storage/overlay/a815ac2e4a767e0671799ad3e90415eb662d179b1343bebb7b77824b75be7607/merged major:0 minor:250 fsType:overlay blockSize:0} overlay_0-252:{mountpoint:/var/lib/containers/storage/overlay/c392eeee1d777d5b70992a1e34906d923cdc52127408705aa82897451a3e74fa/merged major:0 minor:252 fsType:overlay blockSize:0} overlay_0-254:{mountpoint:/var/lib/containers/storage/overlay/92bd80ca690cdb7d6c6a23222f1d5df32e24c7496c58751c94c88626871e9701/merged major:0 minor:254 fsType:overlay blockSize:0} overlay_0-256:{mountpoint:/var/lib/containers/storage/overlay/c1c8201f9a15046b7db3882187218505c0b4a09b3b7c19945033a3d564ddd11b/merged major:0 minor:256 fsType:overlay blockSize:0} overlay_0-258:{mountpoint:/var/lib/containers/storage/overlay/0e74272ce6e1783c1dbaa42aa3ea1c49a8eac14bea683a87c4c8677a3e64baa5/merged major:0 minor:258 fsType:overlay blockSize:0} overlay_0-260:{mountpoint:/var/lib/containers/storage/overlay/958cc7ad5102d5195ffe8af319c5c2245ad3e12865f429ecbbdb30641adc707f/merged major:0 minor:260 fsType:overlay blockSize:0} overlay_0-262:{mountpoint:/var/lib/containers/storage/overlay/30dbe9f4b0d6ed6c13e0f229b47002bf153d95aaa71790124651a987a14310ea/merged major:0 minor:262 fsType:overlay blockSize:0} overlay_0-264:{mountpoint:/var/lib/containers/storage/overlay/78027c948880a7de1d96d92fad1d68b98260eeae93edaaa587e26dd16635fff0/merged major:0 minor:264 fsType:overlay blockSize:0} overlay_0-266:{mountpoint:/var/lib/containers/storage/overlay/13fccb7b4cc030fbae117309c2ae7a1d0f7b92a26c062a7be0ce1fec134ef480/merged major:0 minor:266 fsType:overlay blockSize:0} overlay_0-281:{mountpoint:/var/lib/containers/storage/overlay/7d0c5f3e2e2c9f037cb0b16f1e54fc30e486494614c0769caa7166a77a17b6ad/merged major:0 minor:281 fsType:overlay blockSize:0} overlay_0-283:{mountpoint:/var/lib/containers/storage/overlay/82537ec87df8b973c1cc37ed35c21619a492124809535d25200db6d8c9afba04/merged major:0 minor:283 fsType:overlay blockSize:0} overlay_0-285:{mountpoint:/var/lib/containers/storage/overlay/8ea6e8f82cce9dd82528367d71cd02d83f8fb912f6bf480f086ad7dd038a39cc/merged major:0 minor:285 fsType:overlay blockSize:0} overlay_0-289:{mountpoint:/var/lib/containers/storage/overlay/460d154c6d6120953852847f5b50f81655400bcce5caa5531eca7273581c331c/merged major:0 minor:289 fsType:overlay blockSize:0} overlay_0-291:{mountpoint:/var/lib/containers/storage/overlay/c33fb4d2b2051c94b0e7dbad811cd9e39d4a383c190afebacf3bd64ff3c167b7/merged major:0 minor:291 fsType:overlay blockSize:0} overlay_0-293:{mountpoint:/var/lib/containers/storage/overlay/645006d0a1fffeb3c5afafb7f2e956446dfa6307c90f75bf2368ab7c8f76149a/merged major:0 minor:293 fsType:overlay blockSize:0} overlay_0-299:{mountpoint:/var/lib/containers/storage/overlay/d46381abfcbeded534d79eac966ca7b70d9757edbe43d4a81adb526c1d632223/merged major:0 minor:299 fsType:overlay blockSize:0} overlay_0-301:{mountpoint:/var/lib/containers/storage/overlay/b49286d3eac7f4606ebe43a8e2056bbde55097a881d740ffa81780bc22cb9cb9/merged major:0 minor:301 fsType:overlay blockSize:0} overlay_0-303:{mountpoint:/var/lib/containers/storage/overlay/92affa704b887f3a20f4aeb72f39220d384f0b20de7eae41431441589c9823cd/merged major:0 minor:303 fsType:overlay blockSize:0} overlay_0-305:{mountpoint:/var/lib/containers/storage/overlay/b9e4091f1c04154e3fe133427d38184375919a25aa17dd47fe7360531b5d0f7c/merged major:0 minor:305 fsType:overlay blockSize:0} overlay_0-311:{mountpoint:/var/lib/containers/storage/overlay/268a60df30abe8e8e3307138e3b765571985e04e5b4bb8ec7caf5fa7c877007a/merged major:0 minor:311 fsType:overlay blockSize:0} overlay_0-343:{mountpoint:/var/lib/containers/storage/overlay/456bc6423fd015634b1efa8dec398ec9319a393c14a5a085c2f9804d7d54e508/merged major:0 minor:343 fsType:overlay blockSize:0} overlay_0-345:{mountpoint:/var/lib/containers/storage/overlay/bdf234f1b8bb6d3fa2e1aa93cec6da3a703b83680a63af60e3436a5dddf6ce61/merged major:0 minor:345 fsType:overlay blockSize:0} overlay_0-347:{mountpoint:/var/lib/containers/storage/overlay/db0811ba379b556d04577ecc1c1fa901b2bfc38771289a1f332902c3771d6272/merged major:0 minor:347 fsType:overlay blockSize:0} overlay_0-353:{mountpoint:/var/lib/containers/storage/overlay/c17774a977f63236600fe90b353d430213c48c0c2d40f7192018b5696ae3ed67/merged major:0 minor:353 fsType:overlay blockSize:0} overlay_0-359:{mountpoint:/var/lib/containers/storage/overlay/695c152bf4764c467130524f74563519b3710123b46e5abd20044fdbac5f50e8/merged major:0 minor:359 fsType:overlay blockSize:0} overlay_0-361:{mountpoint:/var/lib/containers/storage/overlay/0584e568993144856d9e305a79667cf181952e8aad9ec2b965eecadb753228b8/merged major:0 minor:361 fsType:overlay blockSize:0} overlay_0-363:{mountpoint:/var/lib/containers/storage/overlay/70117f64621757fe3a52d77c02630c12570d11a47cae20a7abd5f0e4450ad9d2/merged major:0 minor:363 fsType:overlay blockSize:0} overlay_0-369:{mountpoint:/var/lib/containers/storage/overlay/e351d2f8ac3aa3d0c3377ca0c13df4ca06bdca8404194bb434e3e0b7aaac794e/merged major:0 minor:369 fsType:overlay blockSize:0} overlay_0-388:{mountpoint:/var/lib/containers/storage/overlay/039f53122e49f242d94b5efad3cb544d375d0a3dc34ba4b23caec43cc8c0f520/merged major:0 minor:388 fsType:overlay blockSize:0} overlay_0-390:{mountpoint:/var/lib/containers/storage/overlay/b76b673e622b71a19ceebbb53f6402646ecc39c0745792682dab45607f8afc85/merged major:0 minor:390 fsType:overlay blockSize:0} overlay_0-392:{mountpoint:/var/lib/containers/storage/overlay/f254b23b0cf4d317f23947e63ba2ae9ac1cb8b4e819045fec7887f8d9c489a5b/merged major:0 minor:392 fsType:overlay blockSize:0} overlay_0-394:{mountpoint:/var/lib/containers/storage/overlay/11c4928a4f089afbf0e70dbaa07eb612d89adc38656efb4feff0cfde33b54781/merged major:0 minor:394 fsType:overlay blockSize:0} overlay_0-400:{mountpoint:/var/lib/containers/storage/overlay/e7c53130836434d3067061af0d580bc6a9e0b94611bb8656e72f9e097b64e962/merged major:0 minor:400 fsType:overlay blockSize:0} overlay_0-402:{mountpoint:/var/lib/containers/storage/overlay/a62160d448657df454b50d7721a49be672c1de2922ecb9dac56e6ba51bf8f99c/merged major:0 minor:402 fsType:overlay blockSize:0} overlay_0-408:{mountpoint:/var/lib/containers/storage/overlay/362f5c0bb0d1ad23c4d79b5094c276bd25a1b12615823ead9e4fbc02896546f2/merged major:0 minor:408 fsType:overlay blockSize:0} overlay_0-421:{mountpoint:/var/lib/containers/storage/overlay/14c238dec987368ed31661c1c85bbef6babc819c3b0d6b4bb33b21a6ce2a3c28/merged major:0 minor:421 fsType:overlay blockSize:0} overlay_0-423:{mountpoint:/var/lib/containers/storage/overlay/19e45e86fbcc5fc6f6c8b56e27256625a6610d670848fc89fb24aeed8ce6bb16/merged major:0 minor:423 fsType:overlay blockSize:0} overlay_0-425:{mountpoint:/var/lib/containers/storage/overlay/b7fa6b3e0aa368cba4ae7967a35746e6bd8871008ed9889ef82f0eb18463b2bf/merged major:0 minor:425 fsType:overlay blockSize:0} overlay_0-431:{mountpoint:/var/lib/containers/storage/overlay/53ca2755c45d676b09fe9e1cb40ca63bf869ee7fe0ac003be33b2601b981230b/merged major:0 minor:431 fsType:overlay blockSize:0} overlay_0-444:{mountpoint:/var/lib/containers/storage/overlay/b5660e3db7241a4abdeb38d7d665e38ec21504c777fe379b5690b1a835deb22a/merged major:0 minor:444 fsType:overlay blockSize:0} overlay_0-446:{mountpoint:/var/lib/containers/storage/overlay/1c9ae60bd9d104d0c86a321ce1a7d47d0f40bfefc7e1a6ee96d5b54c4a249b52/merged major:0 minor:446 fsType:overlay blockSize:0} overlay_0-448:{mountpoint:/var/lib/containers/storage/overlay/5f758c00dde5515fd36c19eb2e319424000d5edb38ff0918ea084b493e5575bb/merged major:0 minor:448 fsType:overlay blockSize:0} overlay_0-454:{mountpoint:/var/lib/containers/storage/overlay/63e42acab12d0c2ab7a881b6e512702272a5f7f0c1f6547830a89a71f6810b1b/merged major:0 minor:454 fsType:overlay blockSize:0} overlay_0-466:{mountpoint:/var/lib/containers/storage/overlay/e664aabb10a01dbd21e2c714e12fab5e439f7024925f1d368123892413809764/merged major:0 minor:466 fsType:overlay blockSize:0} overlay_0-468:{mountpoint:/var/lib/containers/storage/overlay/bd8876fe76861e1293d6512dda50717a7baa35261c9d2abbd00b93a91935fcff/merged major:0 minor:468 fsType:overlay blockSize:0} overlay_0-470:{mountpoint:/var/lib/containers/storage/overlay/4713d1b2b5215274e3b377f159202b00dc383ec1bdef413a40eea7c33eff91d3/merged major:0 minor:470 fsType:overlay blockSize:0} overlay_0-472:{mountpoint:/var/lib/containers/storage/overlay/be0dccd43a622130196022f165755017f91e4f20e2614f642dd5ffbcad9dcbd6/merged major:0 minor:472 fsType:overlay blockSize:0} overlay_0-477:{mountpoint:/var/lib/containers/storage/overlay/1661d99dfd0cdd56c43d52bdb84335b14bd5c44d996e0260d42177bd45504e9b/merged major:0 minor:477 fsType:overlay blockSize:0} overlay_0-482:{mountpoint:/var/lib/containers/storage/overlay/a3056524e48dae3cb1c92b291f7b11147d58cbde9cf221aa41962a0d6d292201/merged major:0 minor:482 fsType:overlay blockSize:0} overlay_0-488:{mountpoint:/var/lib/containers/storage/overlay/b89a4503fb8b4b3824b2af1314d00ec897060d86d87eb2e9c9bc131b8d5235c6/merged major:0 minor:488 fsType:overlay blockSize:0} overlay_0-493:{mountpoint:/var/lib/containers/storage/overlay/57ddbc1408a797cbd8e111aaf832e041129b344cdaaacacf6b985b39601de66a/merged major:0 minor:493 fsType:overlay blockSize:0} overlay_0-499:{mountpoint:/var/lib/containers/storage/overlay/05baf177f44a8a4af14e465a64780c2304d3f9639e9c3476a898cf0c56925f16/merged major:0 minor:499 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/27f5a7ee2bd6d167b78ceeaf2d6eaccca5e3c1c3a67885240687d9d9e8e8c1c8/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/97b0f1b96b89d14a341acf16641ec1c31ba71012722f8b29a4da5aba3971da54/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-72:{mountpoint:/var/lib/containers/storage/overlay/f69fcb514db8156bfd32ef3beff8db1e11ee902cf1fc5a4d342f99aa90db6950/merged major:0 minor:72 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/bccd58b368fa14dae1f4268f47b5c7ecb719d62e2d92d6812ee2cc9bf9e47ba3/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-76:{mountpoint:/var/lib/containers/storage/overlay/564a5be223bf66d8b4fa8663f0ba1e34384df1d2108cbefc7af733b71fe390e2/merged major:0 minor:76 fsType:overlay blockSize:0} overlay_0-80:{mountpoint:/var/lib/containers/storage/overlay/251d258ef5ddc97f8a5b2752d49e064841a6650fd3dff506316ee7988237da4c/merged major:0 minor:80 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/0d054147a1e69ac4d125276b043537e006a1abb50f23b2e940e489072c4742b7/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-88:{mountpoint:/var/lib/containers/storage/overlay/f56707eb35b6cd8d9f92e6280d82c61d77fcea61f80dd243f794206796bff3f1/merged major:0 minor:88 fsType:overlay blockSize:0} overlay_0-92:{mountpoint:/var/lib/containers/storage/overlay/edcb0e5e57707612e8062488e0f2788b1595a1c6ff8c1c70b51cb8d94fac7f85/merged major:0 minor:92 fsType:overlay blockSize:0} overlay_0-96:{mountpoint:/var/lib/containers/storage/overlay/d3a99e3d2c8d8cc367bcf2f689165bd4c6eec92d7eb3f565608f58ca2c207713/merged major:0 minor:96 fsType:overlay blockSize:0}] Oct 30 18:38:49.593004 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.592379 5894 manager.go:217] Machine: {Timestamp:2026-10-30 18:38:49.59154523 +0000 UTC m=+0.081673045 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103993 MemoryCapacity:33164333056 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2270c7bfbc8ca9ba20f574def71eb6 SystemUUID:ec2270c7-bfbc-8ca9-ba20-f574def71eb6 BootID:46453f11-dfb1-4507-a8f2-56c841df8ab8 Filesystems:[{Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/var/lib/kubelet/pods/6b091344-3cff-4f72-9512-02499eb9a9ef/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:50 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/c89d1725-2dbb-49dc-92b1-218935ce8a74/volumes/kubernetes.io~projected/kube-api-access-fkqwc DeviceMajor:0 DeviceMinor:52 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/d11cbe37ba26264691e609ca8fc29e574c0b34e9a14cc855094fdc0350d02859/userdata/shm DeviceMajor:0 DeviceMinor:386 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:206 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-482 DeviceMajor:0 DeviceMinor:482 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-154 DeviceMajor:0 DeviceMinor:154 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/34a1d011-b30b-49b8-bc75-bfb9dbba3817/volumes/kubernetes.io~projected/kube-api-access-4zfjf DeviceMajor:0 DeviceMinor:203 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-281 DeviceMajor:0 DeviceMinor:281 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca/volumes/kubernetes.io~secret/signing-key DeviceMajor:0 DeviceMinor:355 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-468 DeviceMajor:0 DeviceMinor:468 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-254 DeviceMajor:0 DeviceMinor:254 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/15f5b642-cd06-42ad-9832-dcfe438ea4e5/volumes/kubernetes.io~projected/kube-api-access-6tt86 DeviceMajor:0 DeviceMinor:54 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/4a88dde4-2103-476d-81c4-4b255cff4828/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:223 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-421 DeviceMajor:0 DeviceMinor:421 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/9cd97653-a745-4fd6-9a6c-71aaa742b415/volumes/kubernetes.io~secret/samples-operator-tls DeviceMajor:0 DeviceMinor:211 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ba4f4ec128c62434c0766741f1006e473a81b720369bc4ea6e3a8d1cadcb6c31/userdata/shm DeviceMajor:0 DeviceMinor:464 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-477 DeviceMajor:0 DeviceMinor:477 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6308f380c32bcbf4b59577d164b1513e052edcfddce9b3a9e4e0392784d389ae/userdata/shm DeviceMajor:0 DeviceMinor:233 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/6b091344-3cff-4f72-9512-02499eb9a9ef/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:49 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2797391b-fd3d-4222-82c8-b28a2d16c9e3/volumes/kubernetes.io~projected/kube-api-access-lg2js DeviceMajor:0 DeviceMinor:202 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/4a88dde4-2103-476d-81c4-4b255cff4828/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:210 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b6b8b467a9a9c17d405b5f134e0a4f52618908c014498e12875220ea4be2ae9e/userdata/shm DeviceMajor:0 DeviceMinor:244 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-291 DeviceMajor:0 DeviceMinor:291 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-293 DeviceMajor:0 DeviceMinor:293 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/2797391b-fd3d-4222-82c8-b28a2d16c9e3/volumes/kubernetes.io~secret/cluster-monitoring-operator-tls DeviceMajor:0 DeviceMinor:375 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/2bfeb3c6-d71a-4b5e-96db-198da488f7a9/volumes/kubernetes.io~projected/kube-api-access-qndwx DeviceMajor:0 DeviceMinor:57 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/f7b2298ae1b4cf4ef2cbb4e235848ecc9fdfe1cbcb06169900783b1ecdec7e47/userdata/shm DeviceMajor:0 DeviceMinor:62 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-114 DeviceMajor:0 DeviceMinor:114 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e65f9eb5-9e22-4c76-a1b5-e2250003faa6/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:208 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/28a4b9b1d84bfb15c36875674bcc12c565c22f842974116ffe42f7d4f3c326a0/userdata/shm DeviceMajor:0 DeviceMinor:216 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/67b57c01-e4a3-4ae1-ab6e-9b386e181adb/volumes/kubernetes.io~projected/kube-api-access-ghbwl DeviceMajor:0 DeviceMinor:220 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/63658d25-6d63-42e2-9755-8ca66a322d61/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:232 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-172 DeviceMajor:0 DeviceMinor:172 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/10e2ded499f32c29040bcaac33da17b080ba2bf5682452b99e2e12d16855c1fe/userdata/shm DeviceMajor:0 DeviceMinor:246 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1d9056edc0e80f4fa89a98a908130771563b58b346ba3d202690e1907398462f/userdata/shm DeviceMajor:0 DeviceMinor:380 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/8c9b04e2-b099-4704-8786-b64a8991d0f4/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:460 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/ce6f0c54-70e4-4d45-8bc3-2823a815f0db/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:48 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3180fff2-f289-47e6-a37c-34c5dff50aa2/volumes/kubernetes.io~secret/default-certificate DeviceMajor:0 DeviceMinor:212 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e9aa7c0d77120340e0ef3c531e83d699e403f7825a9c67d1ee67556aa802cc05/userdata/shm DeviceMajor:0 DeviceMinor:238 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-252 DeviceMajor:0 DeviceMinor:252 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/cd195fdf-8022-4279-8eca-15d145d59038/volumes/kubernetes.io~projected/kube-api-access-6l7k8 DeviceMajor:0 DeviceMinor:268 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-303 DeviceMajor:0 DeviceMinor:303 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3180fff2-f289-47e6-a37c-34c5dff50aa2/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:383 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/7a4adea8-c5dc-4a4d-ad54-e985838fa6c5/volumes/kubernetes.io~projected/kube-api-access-w6srj DeviceMajor:0 DeviceMinor:201 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9809304bf3e9a708db1a5b926a67a728cc1e9c0f3308b9f01a9f57dd56787f89/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-423 DeviceMajor:0 DeviceMinor:423 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16582164480 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-264 DeviceMajor:0 DeviceMinor:264 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4a88dde4-2103-476d-81c4-4b255cff4828/volumes/kubernetes.io~projected/registry-tls DeviceMajor:0 DeviceMinor:218 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-146 DeviceMajor:0 DeviceMinor:146 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-158 DeviceMajor:0 DeviceMinor:158 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-96 DeviceMajor:0 DeviceMinor:96 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e035e6d046c09feaf39dad9de2fed20fa7f62fac403a143af5925da680b22ca2/userdata/shm DeviceMajor:0 DeviceMinor:60 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-262 DeviceMajor:0 DeviceMinor:262 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/91b2497b-8770-4c64-9b1f-e2ec8744ccf9/volumes/kubernetes.io~projected/kube-api-access-cbjt2 DeviceMajor:0 DeviceMinor:338 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-390 DeviceMajor:0 DeviceMinor:390 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-470 DeviceMajor:0 DeviceMinor:470 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-499 DeviceMajor:0 DeviceMinor:499 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1def0758feaa34df841ee35ec0c9744f1652ec4e3db9d160b8212fff4fb5ed04/userdata/shm DeviceMajor:0 DeviceMinor:63 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/89271a7f-e9c4-43dd-a320-54900b3fb8b7/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:213 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-311 DeviceMajor:0 DeviceMinor:311 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/290aeefa-831a-4e2d-bc4c-11defdfee2d0/volumes/kubernetes.io~projected/kube-api-access-qwgvn DeviceMajor:0 DeviceMinor:325 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-363 DeviceMajor:0 DeviceMinor:363 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4971600d17965249e566952e3d5de9e09491c9d11640b0f63933dc353ea1ab09/userdata/shm DeviceMajor:0 DeviceMinor:228 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-448 DeviceMajor:0 DeviceMinor:448 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca/volumes/kubernetes.io~projected/kube-api-access-b6f6l DeviceMajor:0 DeviceMinor:356 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-402 DeviceMajor:0 DeviceMinor:402 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4336e440d847cb102b5a84cf5db4e4868879b7a45b75ad04742b4b0303ca407f/userdata/shm DeviceMajor:0 DeviceMinor:419 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/run/containers/storage/overlay-containers/234c59b104002e47f821c4f580fd993ad867affa8bd4ad0a833ceb243b979f03/userdata/shm DeviceMajor:0 DeviceMinor:82 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/4656cde633a452376680ea5749895c5b4884b3df064fd06daafcf44e6f1ff605/userdata/shm DeviceMajor:0 DeviceMinor:86 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/9cd97653-a745-4fd6-9a6c-71aaa742b415/volumes/kubernetes.io~projected/kube-api-access-2c2tj DeviceMajor:0 DeviceMinor:222 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f/volumes/kubernetes.io~projected/kube-api-access-v72l5 DeviceMajor:0 DeviceMinor:239 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-285 DeviceMajor:0 DeviceMinor:285 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-289 DeviceMajor:0 DeviceMinor:289 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16582168576 Type:vfs Inodes:1048576 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-305 DeviceMajor:0 DeviceMinor:305 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-388 DeviceMajor:0 DeviceMinor:388 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-400 DeviceMajor:0 DeviceMinor:400 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-472 DeviceMajor:0 DeviceMinor:472 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/6b091344-3cff-4f72-9512-02499eb9a9ef/volumes/kubernetes.io~projected/kube-api-access-thmjv DeviceMajor:0 DeviceMinor:59 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7ca07043-d7b5-4660-87d4-2a92bf5dacbf/volumes/kubernetes.io~projected/kube-api-access-6srz4 DeviceMajor:0 DeviceMinor:204 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/bb52d69e-0e3f-46b5-8491-a2be5a912d29/volumes/kubernetes.io~projected/kube-api-access-2q9pq DeviceMajor:0 DeviceMinor:226 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/06192cb253d0f89797abd9283cd05e09470c34b4bc601a192fd03ad567b03272/userdata/shm DeviceMajor:0 DeviceMinor:341 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3180fff2-f289-47e6-a37c-34c5dff50aa2/volumes/kubernetes.io~secret/stats-auth DeviceMajor:0 DeviceMinor:209 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-361 DeviceMajor:0 DeviceMinor:361 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-256 DeviceMajor:0 DeviceMinor:256 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/4b9a31e1-73c6-4617-ac72-296e03e928f2/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:47 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/e2455903-bb15-41e2-8d91-51df8775b033/volumes/kubernetes.io~projected/kube-api-access-ztsmk DeviceMajor:0 DeviceMinor:51 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00/volumes/kubernetes.io~projected/kube-api-access-qjh2m DeviceMajor:0 DeviceMinor:55 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2fd69b6a889132d82f4380a7c8a7a16109a844804e2c03a418e94af8a666f6bf/userdata/shm DeviceMajor:0 DeviceMinor:61 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6a71eac430b3a11c70f0218e4d5783578dcf216b23d800a92b73ec44e02196c5/userdata/shm DeviceMajor:0 DeviceMinor:214 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-250 DeviceMajor:0 DeviceMinor:250 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ff0ec68a90647ce926cdb4307ff2be59548cfbdc73a2e046c61eade4003464e8/userdata/shm DeviceMajor:0 DeviceMinor:335 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-76 DeviceMajor:0 DeviceMinor:76 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/e65f9eb5-9e22-4c76-a1b5-e2250003faa6/volumes/kubernetes.io~projected/kube-api-access-k44bp DeviceMajor:0 DeviceMinor:221 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/51d9053b-abef-45c8-bd1a-67343812dcd9/volumes/kubernetes.io~projected/kube-api-access-lvrqw DeviceMajor:0 DeviceMinor:237 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/70efc2349fb099a70d72f8c4332cb650c93cabd5374020a22e3c3ca0c2b662a6/userdata/shm DeviceMajor:0 DeviceMinor:339 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-454 DeviceMajor:0 DeviceMinor:454 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-80 DeviceMajor:0 DeviceMinor:80 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ce6f0c54-70e4-4d45-8bc3-2823a815f0db/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6632869888 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/3180fff2-f289-47e6-a37c-34c5dff50aa2/volumes/kubernetes.io~projected/kube-api-access-n4p6x DeviceMajor:0 DeviceMinor:225 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/756fad8be1817e3079ad90464555d4d70a5101b1568c397ddb5ec583a41bbe42/userdata/shm DeviceMajor:0 DeviceMinor:242 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-359 DeviceMajor:0 DeviceMinor:359 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-369 DeviceMajor:0 DeviceMinor:369 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e5622c316fa0601a6308ab7fced872c54601ccf1c0f8b094ac3870a49d6fc09e/userdata/shm DeviceMajor:0 DeviceMinor:442 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/ce6f0c54-70e4-4d45-8bc3-2823a815f0db/volumes/kubernetes.io~projected/kube-api-access-xmb9d DeviceMajor:0 DeviceMinor:58 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-92 DeviceMajor:0 DeviceMinor:92 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a36f2ae5-c6ca-49cf-a775-82a97e92f2c4/volumes/kubernetes.io~projected/kube-api-access-m5w86 DeviceMajor:0 DeviceMinor:197 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/4a88dde4-2103-476d-81c4-4b255cff4828/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:205 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/67b57c01-e4a3-4ae1-ab6e-9b386e181adb/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:207 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/bdda3f2b9e852331ec036d72d43573711d031903faddb053be331836e4cf3d6d/userdata/shm DeviceMajor:0 DeviceMinor:227 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/7ca07043-d7b5-4660-87d4-2a92bf5dacbf/volumes/kubernetes.io~secret/serving-cert DeviceMajor:0 DeviceMinor:45 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-345 DeviceMajor:0 DeviceMinor:345 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/c135952e-12c3-4a65-bd3d-0d0b1aba49d1/volumes/kubernetes.io~secret/networking-console-plugin-cert DeviceMajor:0 DeviceMinor:382 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-88 DeviceMajor:0 DeviceMinor:88 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7eed8239-b695-41f7-968f-b3a02659b44c/volumes/kubernetes.io~projected/kube-api-access-wqsv7 DeviceMajor:0 DeviceMinor:53 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ccf8d80ee7794e49416a9b3d06822143ece1c2af772de7d1b76b0faa5bbd58d5/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/4a88dde4-2103-476d-81c4-4b255cff4828/volumes/kubernetes.io~projected/kube-api-access-v87g2 DeviceMajor:0 DeviceMinor:219 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5dd77f7e140b9dab12799c0b35a5dc1c3df7894456b2c54abf57f224455ee922/userdata/shm DeviceMajor:0 DeviceMinor:384 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-425 DeviceMajor:0 DeviceMinor:425 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-431 DeviceMajor:0 DeviceMinor:431 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/12afdaecda545f8c43bef504323a532ac5a46dbabd623069abfdac6501c84655/userdata/shm DeviceMajor:0 DeviceMinor:440 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:461 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/510a7c62896ac8e28e4f9b7c8a70f158b31b85c7f39b6faae1447460f7418a12/userdata/shm DeviceMajor:0 DeviceMinor:90 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a590889638ad55563f8b4a4f6a2914ff008c4675b8cc971a553efbb73101bc21/userdata/shm DeviceMajor:0 DeviceMinor:98 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-266 DeviceMajor:0 DeviceMinor:266 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/74990028-c4d9-40d7-b1dc-b45b9f3ccbf2/volumes/kubernetes.io~projected/kube-api-access-xh5hg DeviceMajor:0 DeviceMinor:337 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-394 DeviceMajor:0 DeviceMinor:394 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/89271a7f-e9c4-43dd-a320-54900b3fb8b7/volumes/kubernetes.io~projected/kube-api-access-8blzq DeviceMajor:0 DeviceMinor:224 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e09c65370ab654a610997634985c1f985d1ca8b2f217adcfd54b7bbd0af1af48/userdata/shm DeviceMajor:0 DeviceMinor:235 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-258 DeviceMajor:0 DeviceMinor:258 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-283 DeviceMajor:0 DeviceMinor:283 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/b7ce03394c927b190fb2b2d52042e55bddcbb68628f97ae7b932e5e2266f68d3/userdata/shm DeviceMajor:0 DeviceMinor:357 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-466 DeviceMajor:0 DeviceMinor:466 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/a3d0c2abffcf4a28f3b6343dea6d745d27fd48780411b32639a996672e47416d/userdata/shm DeviceMajor:0 DeviceMinor:94 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/63658d25-6d63-42e2-9755-8ca66a322d61/volumes/kubernetes.io~projected/kube-api-access-7hlr2 DeviceMajor:0 DeviceMinor:240 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-343 DeviceMajor:0 DeviceMinor:343 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-347 DeviceMajor:0 DeviceMinor:347 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-353 DeviceMajor:0 DeviceMinor:353 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-392 DeviceMajor:0 DeviceMinor:392 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/3d5e33b3-69cd-4fc7-8366-14ffa038713e/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:414 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-493 DeviceMajor:0 DeviceMinor:493 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-177 DeviceMajor:0 DeviceMinor:177 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-408 DeviceMajor:0 DeviceMinor:408 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-488 DeviceMajor:0 DeviceMinor:488 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2/volumes/kubernetes.io~projected/kube-api-access-b9hs4 DeviceMajor:0 DeviceMinor:56 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:/run/containers/storage/overlay-containers/53177fd6712175b6d60e8410ffa77d7e8f380168e452c864291dfbcb52a18432/userdata/shm DeviceMajor:0 DeviceMinor:287 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:/var/lib/kubelet/pods/3d5e33b3-69cd-4fc7-8366-14ffa038713e/volumes/kubernetes.io~projected/kube-api-access-4w8p5 DeviceMajor:0 DeviceMinor:330 Capacity:31985733632 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-444 DeviceMajor:0 DeviceMinor:444 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6898926c00a6bae4a7eaa36bbcf4186056d4fdeeb32b72ad64de694e4ff305b3/userdata/shm DeviceMajor:0 DeviceMinor:462 Capacity:67108864 Type:vfs Inodes:4048380 HasInodes:true} {Device:overlay_0-72 DeviceMajor:0 DeviceMinor:72 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-260 DeviceMajor:0 DeviceMinor:260 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-301 DeviceMajor:0 DeviceMinor:301 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-299 DeviceMajor:0 DeviceMinor:299 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-446 DeviceMajor:0 DeviceMinor:446 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:06192cb253d0f89 MacAddress:42:e5:1d:3f:7a:5b Speed:10000 Mtu:8901} {Name:10e2ded499f32c2 MacAddress:b6:39:a5:55:40:bb Speed:10000 Mtu:8901} {Name:12afdaecda545f8 MacAddress:76:0f:64:bc:4b:86 Speed:10000 Mtu:8901} {Name:1d9056edc0e80f4 MacAddress:ba:51:5a:72:cf:df Speed:10000 Mtu:8901} {Name:28a4b9b1d84bfb1 MacAddress:92:17:af:33:5e:06 Speed:10000 Mtu:8901} {Name:4336e440d847cb1 MacAddress:3e:f7:2d:1e:e9:28 Speed:10000 Mtu:8901} {Name:4971600d1796524 MacAddress:2e:c6:97:61:bf:bc Speed:10000 Mtu:8901} {Name:53177fd6712175b MacAddress:9a:33:e9:a4:e2:6e Speed:10000 Mtu:8901} {Name:5dd77f7e140b9da MacAddress:7a:5f:3b:c1:51:ab Speed:10000 Mtu:8901} {Name:6308f380c32bcbf MacAddress:fa:5d:bf:3d:e0:33 Speed:10000 Mtu:8901} {Name:6898926c00a6bae MacAddress:7a:74:c2:04:81:64 Speed:10000 Mtu:8901} {Name:6a71eac430b3a11 MacAddress:1e:b0:03:a5:ea:c8 Speed:10000 Mtu:8901} {Name:70efc2349fb099a MacAddress:7a:e6:e9:0e:12:39 Speed:10000 Mtu:8901} {Name:756fad8be1817e3 MacAddress:9e:45:e4:d4:93:9a Speed:10000 Mtu:8901} {Name:9809304bf3e9a70 MacAddress:0a:73:c4:6f:bc:82 Speed:10000 Mtu:8901} {Name:b6b8b467a9a9c17 MacAddress:f2:dd:1a:83:79:34 Speed:10000 Mtu:8901} {Name:b7ce03394c927b1 MacAddress:22:c1:08:92:67:2a Speed:10000 Mtu:8901} {Name:ba4f4ec128c6243 MacAddress:d6:a7:7e:d3:2c:70 Speed:10000 Mtu:8901} {Name:bdda3f2b9e85233 MacAddress:de:cf:e6:c7:0b:6a Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:66:a7:28:44:0f Speed:0 Mtu:9001} {Name:br-int MacAddress:ae:0e:05:43:27:67 Speed:0 Mtu:8901} {Name:d11cbe37ba26264 MacAddress:b2:a6:da:b2:2c:df Speed:10000 Mtu:8901} {Name:e09c65370ab654a MacAddress:9a:fd:05:e5:fd:2a Speed:10000 Mtu:8901} {Name:e5622c316fa0601 MacAddress:42:e3:10:0d:01:4b Speed:10000 Mtu:8901} {Name:e9aa7c0d7712034 MacAddress:6e:16:66:7b:7d:71 Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:66:a7:28:44:0f Speed:0 Mtu:9001} {Name:ff0ec68a90647ce MacAddress:52:62:a5:a9:36:e6 Speed:10000 Mtu:8901} {Name:genev_sys_6081 MacAddress:52:4a:e8:f9:fe:1c Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:84:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:ae:64:8a:d1:2d:7e Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164333056 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Oct 30 18:38:49.593367 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.593012 5894 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Oct 30 18:38:49.593367 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.593077 5894 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Oct 30 18:38:49.593474 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.593382 5894 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Oct 30 18:38:49.593599 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.593414 5894 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-122.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Oct 30 18:38:49.593685 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.593600 5894 topology_manager.go:143] "Creating topology manager with none policy" Oct 30 18:38:49.593685 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.593614 5894 container_manager_linux.go:311] "Creating device plugin manager" Oct 30 18:38:49.593685 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.593632 5894 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:38:49.593685 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.593654 5894 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Oct 30 18:38:49.593685 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.593682 5894 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Oct 30 18:38:49.593904 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.593892 5894 state_mem.go:41] "Initialized" logger="CPUManager state memory" Oct 30 18:38:49.594026 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.594014 5894 server.go:1235] "Using root directory" path="/var/lib/kubelet" Oct 30 18:38:49.594079 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.594061 5894 kubelet.go:493] "Attempting to sync node with API server" Oct 30 18:38:49.594132 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.594080 5894 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Oct 30 18:38:49.594132 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.594094 5894 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Oct 30 18:38:49.594132 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.594108 5894 kubelet.go:405] "Adding apiserver pod source" Oct 30 18:38:49.594132 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.594120 5894 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Oct 30 18:38:49.595062 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.595046 5894 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 18:38:49.595062 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.595060 5894 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:38:49.595606 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.595564 5894 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Oct 30 18:38:49.599075 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.599060 5894 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Oct 30 18:38:49.599075 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.599074 5894 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Oct 30 18:38:49.599184 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.599176 5894 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Oct 30 18:38:49.599371 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.599360 5894 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Oct 30 18:38:49.599405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.599389 5894 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Oct 30 18:38:49.599492 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.599483 5894 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Oct 30 18:38:49.599523 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.599499 5894 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Oct 30 18:38:49.599523 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.599506 5894 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Oct 30 18:38:49.599584 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.599523 5894 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Oct 30 18:38:49.599584 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.599534 5894 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Oct 30 18:38:49.599584 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.599541 5894 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Oct 30 18:38:49.599584 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.599546 5894 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Oct 30 18:38:49.599584 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.599552 5894 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Oct 30 18:38:49.599584 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.599559 5894 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Oct 30 18:38:49.599584 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.599566 5894 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Oct 30 18:38:49.599773 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.599595 5894 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Oct 30 18:38:49.599773 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.599606 5894 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Oct 30 18:38:49.599773 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.599633 5894 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Oct 30 18:38:49.599773 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.599640 5894 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Oct 30 18:38:49.603490 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.603472 5894 server.go:1263] "Started kubelet" Oct 30 18:38:49.603606 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.603557 5894 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Oct 30 18:38:49.604051 ip-10-0-132-122 systemd[1]: Started Kubernetes Kubelet. Oct 30 18:38:49.604725 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.604330 5894 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Oct 30 18:38:49.604725 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.604436 5894 server_v1.go:49] "podresources" method="list" useActivePods=true Oct 30 18:38:49.604862 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.604764 5894 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Oct 30 18:38:49.605135 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.605117 5894 server.go:317] "Adding debug handlers to kubelet server" Oct 30 18:38:49.609319 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.609287 5894 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Oct 30 18:38:49.609319 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.609294 5894 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Oct 30 18:38:49.609690 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.609364 5894 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-10-29 18:32:39 +0000 UTC" deadline="2028-06-23 03:53:33.316846017 +0000 UTC" Oct 30 18:38:49.609690 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.609401 5894 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14433h14m43.707449312s" Oct 30 18:38:49.609690 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:38:49.609469 5894 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-122.us-west-1.compute.internal\" not found" Oct 30 18:38:49.609690 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.609492 5894 volume_manager.go:309] "The desired_state_of_world populator starts" Oct 30 18:38:49.609690 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.609512 5894 volume_manager.go:311] "Starting Kubelet Volume Manager" Oct 30 18:38:49.610167 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.609771 5894 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Oct 30 18:38:49.618279 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.618258 5894 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Oct 30 18:38:49.618279 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.618278 5894 factory.go:55] Registering systemd factory Oct 30 18:38:49.618490 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.618291 5894 factory.go:223] Registration of the systemd container factory successfully Oct 30 18:38:49.621011 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.619828 5894 factory.go:153] Registering CRI-O factory Oct 30 18:38:49.621011 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.619847 5894 factory.go:223] Registration of the crio container factory successfully Oct 30 18:38:49.621011 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.619866 5894 factory.go:103] Registering Raw factory Oct 30 18:38:49.621011 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.619879 5894 manager.go:1199] Started watching for new ooms in manager Oct 30 18:38:49.621011 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.620183 5894 manager.go:319] Starting recovery of all containers Oct 30 18:38:49.624698 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.624626 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f" volumeName="kubernetes.io/configmap/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-config-volume" seLinuxMountContext="" Oct 30 18:38:49.624698 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.624687 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3d5e33b3-69cd-4fc7-8366-14ffa038713e" volumeName="kubernetes.io/empty-dir/3d5e33b3-69cd-4fc7-8366-14ffa038713e-data-volume" seLinuxMountContext="" Oct 30 18:38:49.624867 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.624703 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00" volumeName="kubernetes.io/projected/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-kube-api-access-qjh2m" seLinuxMountContext="" Oct 30 18:38:49.624867 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.624738 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="67b57c01-e4a3-4ae1-ab6e-9b386e181adb" volumeName="kubernetes.io/configmap/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-trusted-ca" seLinuxMountContext="" Oct 30 18:38:49.624867 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.624753 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6b091344-3cff-4f72-9512-02499eb9a9ef" volumeName="kubernetes.io/empty-dir/6b091344-3cff-4f72-9512-02499eb9a9ef-tmp" seLinuxMountContext="" Oct 30 18:38:49.624867 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.624767 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77f8df88-fa80-42a5-9ac6-33b2fea5d7ca" volumeName="kubernetes.io/configmap/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca-signing-cabundle" seLinuxMountContext="" Oct 30 18:38:49.626233 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626198 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ca07043-d7b5-4660-87d4-2a92bf5dacbf" volumeName="kubernetes.io/empty-dir/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-snapshots" seLinuxMountContext="" Oct 30 18:38:49.626233 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626230 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7eed8239-b695-41f7-968f-b3a02659b44c" volumeName="kubernetes.io/configmap/7eed8239-b695-41f7-968f-b3a02659b44c-iptables-alerter-script" seLinuxMountContext="" Oct 30 18:38:49.626395 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626249 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8c9b04e2-b099-4704-8786-b64a8991d0f4" volumeName="kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret" seLinuxMountContext="" Oct 30 18:38:49.626395 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626268 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="290aeefa-831a-4e2d-bc4c-11defdfee2d0" volumeName="kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-trusted-ca-bundle" seLinuxMountContext="" Oct 30 18:38:49.626395 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626282 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="34a1d011-b30b-49b8-bc75-bfb9dbba3817" volumeName="kubernetes.io/projected/34a1d011-b30b-49b8-bc75-bfb9dbba3817-kube-api-access-4zfjf" seLinuxMountContext="" Oct 30 18:38:49.626395 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626278 5894 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Oct 30 18:38:49.626395 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626295 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a88dde4-2103-476d-81c4-4b255cff4828" volumeName="kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-bound-sa-token" seLinuxMountContext="" Oct 30 18:38:49.626395 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626331 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9cd97653-a745-4fd6-9a6c-71aaa742b415" volumeName="kubernetes.io/configmap/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-config" seLinuxMountContext="" Oct 30 18:38:49.626395 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626345 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a88dde4-2103-476d-81c4-4b255cff4828" volumeName="kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-trusted-ca" seLinuxMountContext="" Oct 30 18:38:49.626395 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626365 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="91b2497b-8770-4c64-9b1f-e2ec8744ccf9" volumeName="kubernetes.io/projected/91b2497b-8770-4c64-9b1f-e2ec8744ccf9-kube-api-access-cbjt2" seLinuxMountContext="" Oct 30 18:38:49.626395 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626379 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="89271a7f-e9c4-43dd-a320-54900b3fb8b7" volumeName="kubernetes.io/configmap/89271a7f-e9c4-43dd-a320-54900b3fb8b7-config" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626398 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3d5e33b3-69cd-4fc7-8366-14ffa038713e" volumeName="kubernetes.io/projected/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-api-access-4w8p5" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626434 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a88dde4-2103-476d-81c4-4b255cff4828" volumeName="kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-registry-certificates" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626446 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce6f0c54-70e4-4d45-8bc3-2823a815f0db" volumeName="kubernetes.io/configmap/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-env-overrides" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626460 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a88dde4-2103-476d-81c4-4b255cff4828" volumeName="kubernetes.io/empty-dir/4a88dde4-2103-476d-81c4-4b255cff4828-ca-trust-extracted" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626475 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3180fff2-f289-47e6-a37c-34c5dff50aa2" volumeName="kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626487 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2797391b-fd3d-4222-82c8-b28a2d16c9e3" volumeName="kubernetes.io/projected/2797391b-fd3d-4222-82c8-b28a2d16c9e3-kube-api-access-lg2js" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626500 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2797391b-fd3d-4222-82c8-b28a2d16c9e3" volumeName="kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626514 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3180fff2-f289-47e6-a37c-34c5dff50aa2" volumeName="kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-default-certificate" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626526 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a88dde4-2103-476d-81c4-4b255cff4828" volumeName="kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-kube-api-access-v87g2" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626540 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="67b57c01-e4a3-4ae1-ab6e-9b386e181adb" volumeName="kubernetes.io/secret/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-serving-cert" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626556 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9cd97653-a745-4fd6-9a6c-71aaa742b415" volumeName="kubernetes.io/projected/9cd97653-a745-4fd6-9a6c-71aaa742b415-kube-api-access-2c2tj" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626592 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3180fff2-f289-47e6-a37c-34c5dff50aa2" volumeName="kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-stats-auth" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626613 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00" volumeName="kubernetes.io/configmap/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-whereabouts-flatfile-configmap" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626642 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a88dde4-2103-476d-81c4-4b255cff4828" volumeName="kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-image-registry-private-configuration" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626654 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4b9a31e1-73c6-4617-ac72-296e03e928f2" volumeName="kubernetes.io/secret/4b9a31e1-73c6-4617-ac72-296e03e928f2-agent-certs" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626668 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8d3a00e6-202e-4ffb-b8e0-67f15d831ae2" volumeName="kubernetes.io/projected/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-kube-api-access-b9hs4" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626683 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="8d3a00e6-202e-4ffb-b8e0-67f15d831ae2" volumeName="kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626698 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce6f0c54-70e4-4d45-8bc3-2823a815f0db" volumeName="kubernetes.io/configmap/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-ovnkube-config" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626711 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2797391b-fd3d-4222-82c8-b28a2d16c9e3" volumeName="kubernetes.io/configmap/2797391b-fd3d-4222-82c8-b28a2d16c9e3-telemetry-config" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626725 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="290aeefa-831a-4e2d-bc4c-11defdfee2d0" volumeName="kubernetes.io/projected/290aeefa-831a-4e2d-bc4c-11defdfee2d0-kube-api-access-qwgvn" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626737 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2bfeb3c6-d71a-4b5e-96db-198da488f7a9" volumeName="kubernetes.io/projected/2bfeb3c6-d71a-4b5e-96db-198da488f7a9-kube-api-access-qndwx" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626750 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a88dde4-2103-476d-81c4-4b255cff4828" volumeName="kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-installation-pull-secrets" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626764 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="67b57c01-e4a3-4ae1-ab6e-9b386e181adb" volumeName="kubernetes.io/configmap/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-config" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626777 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6b091344-3cff-4f72-9512-02499eb9a9ef" volumeName="kubernetes.io/empty-dir/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-tuned" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626791 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="74990028-c4d9-40d7-b1dc-b45b9f3ccbf2" volumeName="kubernetes.io/projected/74990028-c4d9-40d7-b1dc-b45b9f3ccbf2-kube-api-access-xh5hg" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626805 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f" volumeName="kubernetes.io/projected/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-kube-api-access-v72l5" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626818 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="2bfeb3c6-d71a-4b5e-96db-198da488f7a9" volumeName="kubernetes.io/empty-dir/2bfeb3c6-d71a-4b5e-96db-198da488f7a9-tmp-dir" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626830 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3180fff2-f289-47e6-a37c-34c5dff50aa2" volumeName="kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626846 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ca07043-d7b5-4660-87d4-2a92bf5dacbf" volumeName="kubernetes.io/projected/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-kube-api-access-6srz4" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626858 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c89d1725-2dbb-49dc-92b1-218935ce8a74" volumeName="kubernetes.io/configmap/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-daemon-config" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626872 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="cd195fdf-8022-4279-8eca-15d145d59038" volumeName="kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8" seLinuxMountContext="" Oct 30 18:38:49.626877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626886 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e65f9eb5-9e22-4c76-a1b5-e2250003faa6" volumeName="kubernetes.io/configmap/e65f9eb5-9e22-4c76-a1b5-e2250003faa6-config" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626900 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e65f9eb5-9e22-4c76-a1b5-e2250003faa6" volumeName="kubernetes.io/projected/e65f9eb5-9e22-4c76-a1b5-e2250003faa6-kube-api-access-k44bp" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626918 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e65f9eb5-9e22-4c76-a1b5-e2250003faa6" volumeName="kubernetes.io/secret/e65f9eb5-9e22-4c76-a1b5-e2250003faa6-serving-cert" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626933 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="15f5b642-cd06-42ad-9832-dcfe438ea4e5" volumeName="kubernetes.io/configmap/15f5b642-cd06-42ad-9832-dcfe438ea4e5-serviceca" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626946 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="51d9053b-abef-45c8-bd1a-67343812dcd9" volumeName="kubernetes.io/projected/51d9053b-abef-45c8-bd1a-67343812dcd9-kube-api-access-lvrqw" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626960 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63658d25-6d63-42e2-9755-8ca66a322d61" volumeName="kubernetes.io/projected/63658d25-6d63-42e2-9755-8ca66a322d61-kube-api-access-7hlr2" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626973 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="6b091344-3cff-4f72-9512-02499eb9a9ef" volumeName="kubernetes.io/projected/6b091344-3cff-4f72-9512-02499eb9a9ef-kube-api-access-thmjv" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.626989 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ca07043-d7b5-4660-87d4-2a92bf5dacbf" volumeName="kubernetes.io/configmap/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-service-ca-bundle" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627001 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce6f0c54-70e4-4d45-8bc3-2823a815f0db" volumeName="kubernetes.io/projected/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-kube-api-access-xmb9d" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627027 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="15f5b642-cd06-42ad-9832-dcfe438ea4e5" volumeName="kubernetes.io/projected/15f5b642-cd06-42ad-9832-dcfe438ea4e5-kube-api-access-6tt86" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627040 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="290aeefa-831a-4e2d-bc4c-11defdfee2d0" volumeName="kubernetes.io/empty-dir/290aeefa-831a-4e2d-bc4c-11defdfee2d0-archives-path" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627053 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00" volumeName="kubernetes.io/configmap/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-cni-sysctl-allowlist" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627065 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4a88dde4-2103-476d-81c4-4b255cff4828" volumeName="kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627079 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7a4adea8-c5dc-4a4d-ad54-e985838fa6c5" volumeName="kubernetes.io/projected/7a4adea8-c5dc-4a4d-ad54-e985838fa6c5-kube-api-access-w6srj" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627092 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="89271a7f-e9c4-43dd-a320-54900b3fb8b7" volumeName="kubernetes.io/projected/89271a7f-e9c4-43dd-a320-54900b3fb8b7-kube-api-access-8blzq" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627107 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb52d69e-0e3f-46b5-8491-a2be5a912d29" volumeName="kubernetes.io/projected/bb52d69e-0e3f-46b5-8491-a2be5a912d29-kube-api-access-2q9pq" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627120 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c135952e-12c3-4a65-bd3d-0d0b1aba49d1" volumeName="kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627134 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3180fff2-f289-47e6-a37c-34c5dff50aa2" volumeName="kubernetes.io/projected/3180fff2-f289-47e6-a37c-34c5dff50aa2-kube-api-access-n4p6x" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627149 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ca07043-d7b5-4660-87d4-2a92bf5dacbf" volumeName="kubernetes.io/secret/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-serving-cert" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627162 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="89271a7f-e9c4-43dd-a320-54900b3fb8b7" volumeName="kubernetes.io/secret/89271a7f-e9c4-43dd-a320-54900b3fb8b7-serving-cert" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627178 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a36f2ae5-c6ca-49cf-a775-82a97e92f2c4" volumeName="kubernetes.io/projected/a36f2ae5-c6ca-49cf-a775-82a97e92f2c4-kube-api-access-m5w86" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627192 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c135952e-12c3-4a65-bd3d-0d0b1aba49d1" volumeName="kubernetes.io/configmap/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-nginx-conf" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627205 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c89d1725-2dbb-49dc-92b1-218935ce8a74" volumeName="kubernetes.io/configmap/c89d1725-2dbb-49dc-92b1-218935ce8a74-cni-binary-copy" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627230 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="e2455903-bb15-41e2-8d91-51df8775b033" volumeName="kubernetes.io/projected/e2455903-bb15-41e2-8d91-51df8775b033-kube-api-access-ztsmk" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627242 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3d5e33b3-69cd-4fc7-8366-14ffa038713e" volumeName="kubernetes.io/configmap/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-rbac-proxy-cm" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627253 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00" volumeName="kubernetes.io/configmap/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-cni-binary-copy" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627266 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="67b57c01-e4a3-4ae1-ab6e-9b386e181adb" volumeName="kubernetes.io/projected/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-kube-api-access-ghbwl" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627277 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7eed8239-b695-41f7-968f-b3a02659b44c" volumeName="kubernetes.io/projected/7eed8239-b695-41f7-968f-b3a02659b44c-kube-api-access-wqsv7" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627289 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="9cd97653-a745-4fd6-9a6c-71aaa742b415" volumeName="kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627301 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f" volumeName="kubernetes.io/empty-dir/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-tmp-dir" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627313 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce6f0c54-70e4-4d45-8bc3-2823a815f0db" volumeName="kubernetes.io/secret/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-ovn-node-metrics-cert" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627327 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="290aeefa-831a-4e2d-bc4c-11defdfee2d0" volumeName="kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-service-ca-bundle" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627339 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f" volumeName="kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627351 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="3d5e33b3-69cd-4fc7-8366-14ffa038713e" volumeName="kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627363 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="4b9a31e1-73c6-4617-ac72-296e03e928f2" volumeName="kubernetes.io/configmap/4b9a31e1-73c6-4617-ac72-296e03e928f2-konnectivity-ca" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627376 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ca07043-d7b5-4660-87d4-2a92bf5dacbf" volumeName="kubernetes.io/empty-dir/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-tmp" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627388 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="c89d1725-2dbb-49dc-92b1-218935ce8a74" volumeName="kubernetes.io/projected/c89d1725-2dbb-49dc-92b1-218935ce8a74-kube-api-access-fkqwc" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627401 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ce6f0c54-70e4-4d45-8bc3-2823a815f0db" volumeName="kubernetes.io/configmap/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-ovnkube-script-lib" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627414 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="63658d25-6d63-42e2-9755-8ca66a322d61" volumeName="kubernetes.io/secret/63658d25-6d63-42e2-9755-8ca66a322d61-cert" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627429 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77f8df88-fa80-42a5-9ac6-33b2fea5d7ca" volumeName="kubernetes.io/projected/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca-kube-api-access-b6f6l" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627441 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="77f8df88-fa80-42a5-9ac6-33b2fea5d7ca" volumeName="kubernetes.io/secret/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca-signing-key" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627453 5894 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7ca07043-d7b5-4660-87d4-2a92bf5dacbf" volumeName="kubernetes.io/configmap/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-trusted-ca-bundle" seLinuxMountContext="" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627466 5894 reconstruct.go:97] "Volume reconstruction finished" Oct 30 18:38:49.628349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.627474 5894 reconciler.go:29] "Reconciler: start to sync state" Oct 30 18:38:49.639215 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.639191 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 18:38:49.640188 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.640166 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 18:38:49.641951 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.641928 5894 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Oct 30 18:38:49.641951 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.641953 5894 status_manager.go:249] "Starting to sync pod status with apiserver" Oct 30 18:38:49.642082 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.641975 5894 kubelet.go:2521] "Starting kubelet main sync loop" Oct 30 18:38:49.642082 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:38:49.642012 5894 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Oct 30 18:38:49.661039 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.661016 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 18:38:49.661145 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.661053 5894 generic.go:356] "Generic (PLEG): container finished" podID="67b57c01-e4a3-4ae1-ab6e-9b386e181adb" containerID="a69c05894335c282f0d089331d4cc762b19d49ccf1bc60df7aec93b5023cbc6f" exitCode=255 Oct 30 18:38:49.668518 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.668491 5894 generic.go:356] "Generic (PLEG): container finished" podID="44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00" containerID="e98cb7f20d5dfbb0b5945f19f644cdfb55ac33cfc190c81f568065c24658c8f2" exitCode=0 Oct 30 18:38:49.668636 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.668521 5894 generic.go:356] "Generic (PLEG): container finished" podID="44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00" containerID="db6d97685ceccdc5eba78b0c3bb813cbcbe5a2133b6200a9f4f4857837a11354" exitCode=0 Oct 30 18:38:49.668636 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.668535 5894 generic.go:356] "Generic (PLEG): container finished" podID="44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00" containerID="7689046d138217a2310d3b9ceae4f1f251559248df3dc999a1c68e581a996a33" exitCode=0 Oct 30 18:38:49.668636 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.668543 5894 generic.go:356] "Generic (PLEG): container finished" podID="44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00" containerID="52606d8863eaeb63a2849cfaa451481b3c2aec3152be0701052bc3dffb95af6b" exitCode=0 Oct 30 18:38:49.668636 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.668551 5894 generic.go:356] "Generic (PLEG): container finished" podID="44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00" containerID="055a7fa08aea0465eb32fe2b025ed798bdb47927d0a8a05b70f48ac33af074d5" exitCode=0 Oct 30 18:38:49.668636 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.668558 5894 generic.go:356] "Generic (PLEG): container finished" podID="44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00" containerID="080fb3ae1d239c7bc7fd27935922348d1771c73ea19138a2e83f6725cec548d9" exitCode=0 Oct 30 18:38:49.694885 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.694855 5894 generic.go:356] "Generic (PLEG): container finished" podID="7dbda9e04d721f174c9b45daae509010" containerID="d1b97879fe0f3e7c2781898547d40cc6b991e793b9fc595bbc4535ed114c2066" exitCode=0 Oct 30 18:38:49.709554 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:38:49.709525 5894 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-132-122.us-west-1.compute.internal\" not found" Oct 30 18:38:49.741161 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.741135 5894 manager.go:324] Recovery completed Oct 30 18:38:49.742088 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:38:49.742071 5894 kubelet.go:2545] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Oct 30 18:38:49.763013 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.762994 5894 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:38:49.767434 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.767417 5894 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:38:49.767519 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.767446 5894 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:38:49.767519 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.767457 5894 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientPID" Oct 30 18:38:49.770346 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.770321 5894 cpu_manager.go:226] "Starting" policy="none" Oct 30 18:38:49.770346 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.770345 5894 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Oct 30 18:38:49.770486 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.770368 5894 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Oct 30 18:38:49.770586 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.770558 5894 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Oct 30 18:38:49.770635 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.770592 5894 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Oct 30 18:38:49.770635 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.770617 5894 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Oct 30 18:38:49.770635 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.770626 5894 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Oct 30 18:38:49.770635 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.770634 5894 policy_none.go:50] "Start" Oct 30 18:38:49.770786 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.770658 5894 memory_manager.go:187] "Starting memorymanager" policy="None" Oct 30 18:38:49.770786 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.770672 5894 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Oct 30 18:38:49.772192 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.772169 5894 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:38:49.773726 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.773705 5894 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:38:49.773806 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.773774 5894 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:38:49.773806 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.773787 5894 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientPID" Oct 30 18:38:49.779396 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.779380 5894 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Oct 30 18:38:49.779445 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.779403 5894 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Oct 30 18:38:49.779445 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.779413 5894 policy_none.go:44] "Start" Oct 30 18:38:49.784456 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.784428 5894 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Oct 30 18:38:49.787951 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.787914 5894 manager.go:341] "Starting Device Plugin manager" Oct 30 18:38:49.787951 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:38:49.787938 5894 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Oct 30 18:38:49.787951 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.787946 5894 server.go:87] "Starting device plugin registration server" Oct 30 18:38:49.788275 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.788262 5894 eviction_manager.go:194] "Eviction manager: starting control loop" Oct 30 18:38:49.788323 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.788277 5894 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Oct 30 18:38:49.788398 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.788381 5894 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Oct 30 18:38:49.788537 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.788495 5894 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Oct 30 18:38:49.788620 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.788538 5894 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Oct 30 18:38:49.788620 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.788545 5894 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Oct 30 18:38:49.788709 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.788616 5894 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-10-30T18:38:49.788527192Z","UUID":"d4d77cdc-8a78-44b5-87e9-11af74f3026e","Handler":null,"Name":"","Endpoint":""} Oct 30 18:38:49.790296 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:38:49.790268 5894 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Oct 30 18:38:49.790397 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:38:49.790309 5894 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-122.us-west-1.compute.internal\" not found" Oct 30 18:38:49.790470 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.790457 5894 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Oct 30 18:38:49.790515 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.790489 5894 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Oct 30 18:38:49.804455 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.804432 5894 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:38:49.804455 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.804438 5894 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:38:49.804644 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.804631 5894 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Oct 30 18:38:49.852883 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.852830 5894 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:38:49.889043 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.889020 5894 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Oct 30 18:38:49.890195 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.890179 5894 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientMemory" Oct 30 18:38:49.890263 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.890210 5894 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Oct 30 18:38:49.890263 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.890222 5894 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-132-122.us-west-1.compute.internal" event="NodeHasSufficientPID" Oct 30 18:38:49.890332 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.890265 5894 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:49.902852 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.902824 5894 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Oct 30 18:38:49.942852 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.942816 5894 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal"] Oct 30 18:38:49.943539 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.943522 5894 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:49.943690 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.943646 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal" event={"ID":"99a39564afa8d640f032873b7cd4dd97","Type":"ContainerStarted","Data":"896b65bf8cde3f532d8daf6aa7f29733d8d4dc48fe4b8f554450a73ad4547ea4"} Oct 30 18:38:49.943733 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.943706 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal" event={"ID":"99a39564afa8d640f032873b7cd4dd97","Type":"ContainerStarted","Data":"e035e6d046c09feaf39dad9de2fed20fa7f62fac403a143af5925da680b22ca2"} Oct 30 18:38:49.943764 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.943734 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" event={"ID":"7dbda9e04d721f174c9b45daae509010","Type":"ContainerStarted","Data":"243b12fb8743bcef79a01a360f8d101905cf6b0827ef70b5dbf787f4df1de801"} Oct 30 18:38:49.943764 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.943737 5894 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:49.943764 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.943748 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" event={"ID":"7dbda9e04d721f174c9b45daae509010","Type":"ContainerDied","Data":"d1b97879fe0f3e7c2781898547d40cc6b991e793b9fc595bbc4535ed114c2066"} Oct 30 18:38:49.943764 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.943761 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" event={"ID":"7dbda9e04d721f174c9b45daae509010","Type":"ContainerStarted","Data":"1def0758feaa34df841ee35ec0c9744f1652ec4e3db9d160b8212fff4fb5ed04"} Oct 30 18:38:49.998482 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.998456 5894 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:49.998599 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:49.998522 5894 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:50.007083 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.007065 5894 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:38:50.007158 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:38:50.007109 5894 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:50.007408 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.007394 5894 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:38:50.007443 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:38:50.007429 5894 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:50.106241 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.106182 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7dbda9e04d721f174c9b45daae509010-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal\" (UID: \"7dbda9e04d721f174c9b45daae509010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:50.106241 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.106210 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7dbda9e04d721f174c9b45daae509010-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal\" (UID: \"7dbda9e04d721f174c9b45daae509010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:50.106241 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.106228 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/99a39564afa8d640f032873b7cd4dd97-config\") pod \"kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal\" (UID: \"99a39564afa8d640f032873b7cd4dd97\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:50.206682 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.206652 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7dbda9e04d721f174c9b45daae509010-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal\" (UID: \"7dbda9e04d721f174c9b45daae509010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:50.206682 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.206683 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/99a39564afa8d640f032873b7cd4dd97-config\") pod \"kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal\" (UID: \"99a39564afa8d640f032873b7cd4dd97\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:50.206853 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.206701 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7dbda9e04d721f174c9b45daae509010-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal\" (UID: \"7dbda9e04d721f174c9b45daae509010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:50.206853 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.206720 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7dbda9e04d721f174c9b45daae509010-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal\" (UID: \"7dbda9e04d721f174c9b45daae509010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:50.206853 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.206735 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7dbda9e04d721f174c9b45daae509010-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal\" (UID: \"7dbda9e04d721f174c9b45daae509010\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:50.206853 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.206747 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/99a39564afa8d640f032873b7cd4dd97-config\") pod \"kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal\" (UID: \"99a39564afa8d640f032873b7cd4dd97\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:50.594814 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.594785 5894 apiserver.go:52] "Watching apiserver" Oct 30 18:38:50.698170 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.698140 5894 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Oct 30 18:38:50.698884 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.698856 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fdrkw","kube-system/kas-connection-checker-685895c7c9-mm49z","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal","openshift-kube-storage-version-migrator/migrator-66df5d496b-fjdk5","kube-system/konnectivity-agent-bchfv","openshift-dns/dns-default-kkzsh","openshift-console-operator/console-operator-7cd56b8dbc-gg799","openshift-insights/insights-runtime-extractor-5bn4v","openshift-image-registry/image-registry-7f6754678-55bz6","openshift-network-operator/iptables-alerter-lnjwb","openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb","openshift-network-diagnostics/network-check-source-6f78d59f99-jwxvn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v","openshift-network-diagnostics/network-check-target-95g7n","openshift-multus/multus-m5xnp","openshift-ovn-kubernetes/ovnkube-node-ls845","kube-system/kas-connection-checker-685895c7c9-vwws2","openshift-multus/network-metrics-daemon-2s5rm","openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5","openshift-kube-storage-version-migrator/migrator-66df5d496b-j75jm","openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp","openshift-cluster-node-tuning-operator/tuned-mc5j7","openshift-insights/insights-operator-7c4986d449-6kk4m","openshift-service-ca/service-ca-5fffc97bdf-t2jzr","kube-system/kas-connection-checker-685895c7c9-pvzsh","openshift-insights/periodic-gathering-jmjn6-ghcr6","openshift-ingress/router-default-797b6dc769-r2cdl","openshift-network-console/networking-console-plugin-86c478bb88-hqzbs","openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4z4ss","openshift-dns/node-resolver-5mmnd","kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal","openshift-image-registry/node-ca-kjn4r","openshift-multus/multus-additional-cni-plugins-bh8c5","openshift-monitoring/cluster-monitoring-operator-884576c57-td46q","kube-system/global-pull-secret-syncer-hmzzs"] Oct 30 18:38:50.721204 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.721181 5894 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:50.721339 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.721286 5894 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:50.754141 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.754108 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.754307 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.754188 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Oct 30 18:38:50.754466 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.754450 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.754531 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.754506 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Oct 30 18:38:50.754640 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.754619 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.754701 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.754625 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.754769 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.754717 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.754846 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.754831 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.755137 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.754989 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.755137 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755029 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-74dcr\"" Oct 30 18:38:50.755137 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755082 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.755137 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755100 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.755137 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755106 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.755338 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755232 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Oct 30 18:38:50.755338 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755244 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-mrtvb\"" Oct 30 18:38:50.755338 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755295 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-config\"" Oct 30 18:38:50.755338 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755326 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.755447 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755366 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Oct 30 18:38:50.755705 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755682 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.755705 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755693 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Oct 30 18:38:50.755855 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755823 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.755903 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755863 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Oct 30 18:38:50.755903 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755866 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Oct 30 18:38:50.755903 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755885 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.756038 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755917 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.756038 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755950 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-hrpgl\"" Oct 30 18:38:50.756038 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755962 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Oct 30 18:38:50.756038 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755954 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.756038 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.755956 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Oct 30 18:38:50.756038 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.756016 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mn8z9\"" Oct 30 18:38:50.756251 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.756055 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Oct 30 18:38:50.756251 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.756071 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Oct 30 18:38:50.756251 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.756132 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Oct 30 18:38:50.756251 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.756189 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Oct 30 18:38:50.756251 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.756247 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-qqgkz\"" Oct 30 18:38:50.756640 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.756626 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Oct 30 18:38:50.756707 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.756697 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-n2msw\"" Oct 30 18:38:50.756773 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.756760 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Oct 30 18:38:50.756843 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.756829 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-kk5cx\"" Oct 30 18:38:50.803274 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.803249 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Oct 30 18:38:50.803777 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.803755 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.804132 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.804114 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Oct 30 18:38:50.809513 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.809495 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c2tj\" (UniqueName: \"kubernetes.io/projected/9cd97653-a745-4fd6-9a6c-71aaa742b415-kube-api-access-2c2tj\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:50.809569 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.809523 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:38:50.809569 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.809543 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2797391b-fd3d-4222-82c8-b28a2d16c9e3-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:50.809569 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.809564 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7eed8239-b695-41f7-968f-b3a02659b44c-host-slash\") pod \"iptables-alerter-lnjwb\" (UID: \"7eed8239-b695-41f7-968f-b3a02659b44c\") " pod="openshift-network-operator/iptables-alerter-lnjwb" Oct 30 18:38:50.809712 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.809654 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b091344-3cff-4f72-9512-02499eb9a9ef-tmp\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.809712 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.809704 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-hqzbs\" (UID: \"c135952e-12c3-4a65-bd3d-0d0b1aba49d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" Oct 30 18:38:50.809776 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.809768 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-modprobe-d\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.809817 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.809784 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-node-log\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.809817 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.809802 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l7k8\" (UniqueName: \"kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8\") pod \"network-check-target-95g7n\" (UID: \"cd195fdf-8022-4279-8eca-15d145d59038\") " pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:38:50.809943 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.809819 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k44bp\" (UniqueName: \"kubernetes.io/projected/e65f9eb5-9e22-4c76-a1b5-e2250003faa6-kube-api-access-k44bp\") pod \"service-ca-operator-544d749c8c-xt7x5\" (UID: \"e65f9eb5-9e22-4c76-a1b5-e2250003faa6\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" Oct 30 18:38:50.809943 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.809843 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2bfeb3c6-d71a-4b5e-96db-198da488f7a9-hosts-file\") pod \"node-resolver-5mmnd\" (UID: \"2bfeb3c6-d71a-4b5e-96db-198da488f7a9\") " pod="openshift-dns/node-resolver-5mmnd" Oct 30 18:38:50.809943 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.809900 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hqzbs\" (UID: \"c135952e-12c3-4a65-bd3d-0d0b1aba49d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" Oct 30 18:38:50.809943 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.809922 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-var-lib-openvswitch\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.810129 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.809945 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8c9b04e2-b099-4704-8786-b64a8991d0f4-dbus\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:38:50.810129 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.809974 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqsv7\" (UniqueName: \"kubernetes.io/projected/7eed8239-b695-41f7-968f-b3a02659b44c-kube-api-access-wqsv7\") pod \"iptables-alerter-lnjwb\" (UID: \"7eed8239-b695-41f7-968f-b3a02659b44c\") " pod="openshift-network-operator/iptables-alerter-lnjwb" Oct 30 18:38:50.810129 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.810000 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-device-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.810129 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.810002 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/6b091344-3cff-4f72-9512-02499eb9a9ef-tmp\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.810129 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.810096 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/2797391b-fd3d-4222-82c8-b28a2d16c9e3-telemetry-config\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:50.810322 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.810198 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9hs4\" (UniqueName: \"kubernetes.io/projected/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-kube-api-access-b9hs4\") pod \"network-metrics-daemon-2s5rm\" (UID: \"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2\") " pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:38:50.810322 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.810223 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-run-k8s-cni-cncf-io\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.810322 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.810243 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-cni-binary-copy\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:38:50.810425 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.810401 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thmjv\" (UniqueName: \"kubernetes.io/projected/6b091344-3cff-4f72-9512-02499eb9a9ef-kube-api-access-thmjv\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.810460 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.810443 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-run-netns\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.810498 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.810484 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmb9d\" (UniqueName: \"kubernetes.io/projected/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-kube-api-access-xmb9d\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.810618 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.810602 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-bound-sa-token\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:50.810653 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.810630 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-daemon-config\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.810761 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.810739 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v87g2\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-kube-api-access-v87g2\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:50.810795 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.810785 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:50.810829 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.810813 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-socket-dir-parent\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.810861 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.810854 5894 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Oct 30 18:38:50.810890 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.810858 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qndwx\" (UniqueName: \"kubernetes.io/projected/2bfeb3c6-d71a-4b5e-96db-198da488f7a9-kube-api-access-qndwx\") pod \"node-resolver-5mmnd\" (UID: \"2bfeb3c6-d71a-4b5e-96db-198da488f7a9\") " pod="openshift-dns/node-resolver-5mmnd" Oct 30 18:38:50.810970 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.810953 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs\") pod \"network-metrics-daemon-2s5rm\" (UID: \"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2\") " pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:38:50.811007 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.810993 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c89d1725-2dbb-49dc-92b1-218935ce8a74-cni-binary-copy\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.811040 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811020 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-hostroot\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.811075 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811037 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-config\" (UniqueName: \"kubernetes.io/configmap/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-config\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:50.811075 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811044 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-var-lib-kubelet\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.811075 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811068 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-run-ovn\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.811167 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811097 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-cni-dir\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.811167 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811121 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-sysctl-conf\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.811231 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811160 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-run-systemd\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.811231 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811198 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-registry-certificates\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:50.811295 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811238 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8c9b04e2-b099-4704-8786-b64a8991d0f4-kubelet-config\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:38:50.811295 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811264 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-kubernetes\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.811295 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811280 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-systemd-units\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.811386 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811295 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-slash\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.811386 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811332 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-cni-netd\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.811386 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811369 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.811516 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811392 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-installation-pull-secrets\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:50.811516 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811426 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-sysctl-d\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.811516 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811481 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-tuned\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.811516 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811506 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15f5b642-cd06-42ad-9832-dcfe438ea4e5-host\") pod \"node-ca-kjn4r\" (UID: \"15f5b642-cd06-42ad-9832-dcfe438ea4e5\") " pod="openshift-image-registry/node-ca-kjn4r" Oct 30 18:38:50.811653 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811533 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:50.811653 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811548 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-tuned\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.811653 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811586 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:50.811742 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811674 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjh2m\" (UniqueName: \"kubernetes.io/projected/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-kube-api-access-qjh2m\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:38:50.811778 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811770 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-run-openvswitch\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.811807 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811778 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/2797391b-fd3d-4222-82c8-b28a2d16c9e3-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:50.811842 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811787 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-var-lib-kubelet\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.811873 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811851 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-etc-selinux\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.811909 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811867 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/9cd97653-a745-4fd6-9a6c-71aaa742b415-samples-operator-tls\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:50.811909 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811901 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/89271a7f-e9c4-43dd-a320-54900b3fb8b7-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-689xp\" (UID: \"89271a7f-e9c4-43dd-a320-54900b3fb8b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" Oct 30 18:38:50.811980 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811935 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zfjf\" (UniqueName: \"kubernetes.io/projected/34a1d011-b30b-49b8-bc75-bfb9dbba3817-kube-api-access-4zfjf\") pod \"kas-connection-checker-685895c7c9-vwws2\" (UID: \"34a1d011-b30b-49b8-bc75-bfb9dbba3817\") " pod="kube-system/kas-connection-checker-685895c7c9-vwws2" Oct 30 18:38:50.812011 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.811975 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:38:50.812039 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812012 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-run\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.812091 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812073 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-host\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.812134 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812078 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/89271a7f-e9c4-43dd-a320-54900b3fb8b7-config\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-689xp\" (UID: \"89271a7f-e9c4-43dd-a320-54900b3fb8b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" Oct 30 18:38:50.812134 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812121 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-sys-fs\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.812190 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812160 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:50.812227 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812189 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4p6x\" (UniqueName: \"kubernetes.io/projected/3180fff2-f289-47e6-a37c-34c5dff50aa2-kube-api-access-n4p6x\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:50.812267 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812254 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-sys\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.812326 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812314 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3180fff2-f289-47e6-a37c-34c5dff50aa2-service-ca-bundle\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:50.812359 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812340 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:50.812392 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812371 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6srz4\" (UniqueName: \"kubernetes.io/projected/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-kube-api-access-6srz4\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:50.812421 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812390 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/89271a7f-e9c4-43dd-a320-54900b3fb8b7-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-689xp\" (UID: \"89271a7f-e9c4-43dd-a320-54900b3fb8b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" Oct 30 18:38:50.812464 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812446 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-ovnkube-config\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.812560 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812544 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8blzq\" (UniqueName: \"kubernetes.io/projected/89271a7f-e9c4-43dd-a320-54900b3fb8b7-kube-api-access-8blzq\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-689xp\" (UID: \"89271a7f-e9c4-43dd-a320-54900b3fb8b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" Oct 30 18:38:50.812621 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812583 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63658d25-6d63-42e2-9755-8ca66a322d61-cert\") pod \"ingress-canary-fdrkw\" (UID: \"63658d25-6d63-42e2-9755-8ca66a322d61\") " pod="openshift-ingress-canary/ingress-canary-fdrkw" Oct 30 18:38:50.812621 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812603 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b9a31e1-73c6-4617-ac72-296e03e928f2-konnectivity-ca\") pod \"konnectivity-agent-bchfv\" (UID: \"4b9a31e1-73c6-4617-ac72-296e03e928f2\") " pod="kube-system/konnectivity-agent-bchfv" Oct 30 18:38:50.812712 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812628 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-tmp\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:50.812712 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812653 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-serving-cert\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:50.812813 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812715 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-etc-kubernetes\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.812813 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812723 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-tmp\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:50.812813 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812745 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6srj\" (UniqueName: \"kubernetes.io/projected/7a4adea8-c5dc-4a4d-ad54-e985838fa6c5-kube-api-access-w6srj\") pod \"kas-connection-checker-685895c7c9-mm49z\" (UID: \"7a4adea8-c5dc-4a4d-ad54-e985838fa6c5\") " pod="kube-system/kas-connection-checker-685895c7c9-mm49z" Oct 30 18:38:50.812813 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812788 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/89271a7f-e9c4-43dd-a320-54900b3fb8b7-serving-cert\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-689xp\" (UID: \"89271a7f-e9c4-43dd-a320-54900b3fb8b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" Oct 30 18:38:50.812813 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812795 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-default-certificate\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:50.813025 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812841 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-var-lib-cni-multus\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.813025 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812873 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-conf-dir\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.813025 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812912 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghbwl\" (UniqueName: \"kubernetes.io/projected/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-kube-api-access-ghbwl\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:50.813025 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812958 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:38:50.813025 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.812976 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-default-certificate\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:50.813025 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813004 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bfeb3c6-d71a-4b5e-96db-198da488f7a9-tmp-dir\") pod \"node-resolver-5mmnd\" (UID: \"2bfeb3c6-d71a-4b5e-96db-198da488f7a9\") " pod="openshift-dns/node-resolver-5mmnd" Oct 30 18:38:50.813255 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813067 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2bfeb3c6-d71a-4b5e-96db-198da488f7a9-tmp-dir\") pod \"node-resolver-5mmnd\" (UID: \"2bfeb3c6-d71a-4b5e-96db-198da488f7a9\") " pod="openshift-dns/node-resolver-5mmnd" Oct 30 18:38:50.813255 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813094 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-etc-openvswitch\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.813255 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813113 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-serving-cert\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:50.813255 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813134 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-ovn-node-metrics-cert\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.813255 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813206 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-socket-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.813486 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813253 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-registration-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.813486 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813289 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ztsmk\" (UniqueName: \"kubernetes.io/projected/e2455903-bb15-41e2-8d91-51df8775b033-kube-api-access-ztsmk\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.813486 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813316 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-run-netns\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.813486 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813344 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6tt86\" (UniqueName: \"kubernetes.io/projected/15f5b642-cd06-42ad-9832-dcfe438ea4e5-kube-api-access-6tt86\") pod \"node-ca-kjn4r\" (UID: \"15f5b642-cd06-42ad-9832-dcfe438ea4e5\") " pod="openshift-image-registry/node-ca-kjn4r" Oct 30 18:38:50.813486 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813370 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-serving-cert\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:50.813486 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813422 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-sysconfig\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.813486 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813446 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-log-socket\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.813759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813493 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-run-ovn-kubernetes\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.813759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813518 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5w86\" (UniqueName: \"kubernetes.io/projected/a36f2ae5-c6ca-49cf-a775-82a97e92f2c4-kube-api-access-m5w86\") pod \"kas-connection-checker-685895c7c9-pvzsh\" (UID: \"a36f2ae5-c6ca-49cf-a775-82a97e92f2c4\") " pod="kube-system/kas-connection-checker-685895c7c9-pvzsh" Oct 30 18:38:50.813759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813548 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-env-overrides\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.813759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813600 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a88dde4-2103-476d-81c4-4b255cff4828-ca-trust-extracted\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:50.813759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813621 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-serving-cert\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:50.813759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813629 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-cnibin\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:38:50.813759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813656 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-ovnkube-script-lib\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.813759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813692 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2q9pq\" (UniqueName: \"kubernetes.io/projected/bb52d69e-0e3f-46b5-8491-a2be5a912d29-kube-api-access-2q9pq\") pod \"volume-data-source-validator-58c9f5f46d-4z4ss\" (UID: \"bb52d69e-0e3f-46b5-8491-a2be5a912d29\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4z4ss" Oct 30 18:38:50.813759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813719 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lg2js\" (UniqueName: \"kubernetes.io/projected/2797391b-fd3d-4222-82c8-b28a2d16c9e3-kube-api-access-lg2js\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:50.813759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813725 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a88dde4-2103-476d-81c4-4b255cff4828-ca-trust-extracted\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:50.813759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813743 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-system-cni-dir\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:38:50.813759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813759 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-trusted-ca\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:50.814249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813774 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-snapshots\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:50.814249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813790 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fkqwc\" (UniqueName: \"kubernetes.io/projected/c89d1725-2dbb-49dc-92b1-218935ce8a74-kube-api-access-fkqwc\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.814249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813805 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e65f9eb5-9e22-4c76-a1b5-e2250003faa6-config\") pod \"service-ca-operator-544d749c8c-xt7x5\" (UID: \"e65f9eb5-9e22-4c76-a1b5-e2250003faa6\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" Oct 30 18:38:50.814249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813849 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hlr2\" (UniqueName: \"kubernetes.io/projected/63658d25-6d63-42e2-9755-8ca66a322d61-kube-api-access-7hlr2\") pod \"ingress-canary-fdrkw\" (UID: \"63658d25-6d63-42e2-9755-8ca66a322d61\") " pod="openshift-ingress-canary/ingress-canary-fdrkw" Oct 30 18:38:50.814249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813883 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-var-lib-cni-bin\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.814249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813901 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-cni-bin\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.814249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813932 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-image-registry-private-configuration\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:50.814249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813961 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:38:50.814249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.813991 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b9a31e1-73c6-4617-ac72-296e03e928f2-agent-certs\") pod \"konnectivity-agent-bchfv\" (UID: \"4b9a31e1-73c6-4617-ac72-296e03e928f2\") " pod="kube-system/konnectivity-agent-bchfv" Oct 30 18:38:50.814249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814030 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvrqw\" (UniqueName: \"kubernetes.io/projected/51d9053b-abef-45c8-bd1a-67343812dcd9-kube-api-access-lvrqw\") pod \"network-check-source-6f78d59f99-jwxvn\" (UID: \"51d9053b-abef-45c8-bd1a-67343812dcd9\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-jwxvn" Oct 30 18:38:50.814249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814151 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-snapshots\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:50.814249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814159 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:50.814249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814176 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-trusted-ca\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:50.814249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814163 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e65f9eb5-9e22-4c76-a1b5-e2250003faa6-config\") pod \"service-ca-operator-544d749c8c-xt7x5\" (UID: \"e65f9eb5-9e22-4c76-a1b5-e2250003faa6\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" Oct 30 18:38:50.814249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814194 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-trusted-ca\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:50.814249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814226 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-run-multus-certs\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.814249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814257 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e65f9eb5-9e22-4c76-a1b5-e2250003faa6-serving-cert\") pod \"service-ca-operator-544d749c8c-xt7x5\" (UID: \"e65f9eb5-9e22-4c76-a1b5-e2250003faa6\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" Oct 30 18:38:50.814759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814288 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-config\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:50.814759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814321 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7eed8239-b695-41f7-968f-b3a02659b44c-iptables-alerter-script\") pod \"iptables-alerter-lnjwb\" (UID: \"7eed8239-b695-41f7-968f-b3a02659b44c\") " pod="openshift-network-operator/iptables-alerter-lnjwb" Oct 30 18:38:50.814759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814337 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-stats-auth\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:50.814759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814354 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-cnibin\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.814759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814369 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-kubelet\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.814759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814386 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.814759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814409 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/15f5b642-cd06-42ad-9832-dcfe438ea4e5-serviceca\") pod \"node-ca-kjn4r\" (UID: \"15f5b642-cd06-42ad-9832-dcfe438ea4e5\") " pod="openshift-image-registry/node-ca-kjn4r" Oct 30 18:38:50.814759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814425 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-lib-modules\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.814759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814441 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-service-ca-bundle\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:50.814759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814456 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-system-cni-dir\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.814759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814471 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-systemd\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.814759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814452 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-trusted-ca-bundle\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:50.814759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814651 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e65f9eb5-9e22-4c76-a1b5-e2250003faa6-serving-cert\") pod \"service-ca-operator-544d749c8c-xt7x5\" (UID: \"e65f9eb5-9e22-4c76-a1b5-e2250003faa6\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" Oct 30 18:38:50.814759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814680 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-config\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:50.814759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814736 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:50.815140 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814873 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-stats-auth\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:50.815140 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.814940 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/3180fff2-f289-47e6-a37c-34c5dff50aa2-metrics-certs\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:50.815140 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.815009 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-service-ca-bundle\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:50.816695 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.816682 5894 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:38:50.816733 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:38:50.816720 5894 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:50.817011 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.817000 5894 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Oct 30 18:38:50.817044 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:38:50.817031 5894 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-122.us-west-1.compute.internal" Oct 30 18:38:50.851758 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.851702 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Oct 30 18:38:50.851848 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.851766 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.851848 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.851829 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Oct 30 18:38:50.851848 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.851831 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.851977 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.851898 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Oct 30 18:38:50.852147 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.852111 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.852221 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.852160 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.852221 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.852165 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.852221 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.852171 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.852322 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.852239 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Oct 30 18:38:50.852322 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.852247 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Oct 30 18:38:50.852386 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.852325 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.852386 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.852371 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.852448 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.852395 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.852448 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.852397 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.852594 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.852562 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.852594 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.852564 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Oct 30 18:38:50.852707 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.852597 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Oct 30 18:38:50.852707 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.852565 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.852707 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.852623 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.852821 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.852718 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.852821 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.852762 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-ovnkube-config\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.852821 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.852778 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.855214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.853357 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/4b9a31e1-73c6-4617-ac72-296e03e928f2-konnectivity-ca\") pod \"konnectivity-agent-bchfv\" (UID: \"4b9a31e1-73c6-4617-ac72-296e03e928f2\") " pod="kube-system/konnectivity-agent-bchfv" Oct 30 18:38:50.855214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.853449 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Oct 30 18:38:50.855214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.853632 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.855214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.853885 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Oct 30 18:38:50.855214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.853894 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Oct 30 18:38:50.855214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.854170 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.855214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.854246 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:38:50.855214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.854305 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.855214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.854490 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Oct 30 18:38:50.855214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.854778 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Oct 30 18:38:50.855214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.854979 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.855214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.855091 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.855214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.854983 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Oct 30 18:38:50.855214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.855221 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.855896 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.855299 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/15f5b642-cd06-42ad-9832-dcfe438ea4e5-serviceca\") pod \"node-ca-kjn4r\" (UID: \"15f5b642-cd06-42ad-9832-dcfe438ea4e5\") " pod="openshift-image-registry/node-ca-kjn4r" Oct 30 18:38:50.855896 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.855601 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-env-overrides\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.856254 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.856290 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.856359 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.856523 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.856606 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.856621 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.856637 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bhm78\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.856702 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-dx4x5\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.856720 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.856739 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.856813 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2529f\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.856829 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-q8mbb\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.856851 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-m9jbf\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.856933 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-lp9z6\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.856948 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-kbv4s\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.856997 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.857001 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.857019 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.857031 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-xlrq8\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.857097 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-b7gws\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.857123 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-xhgds\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.857165 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-9gxzb\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.857209 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.857245 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gk56f\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.857284 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.857306 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-476j6\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.857358 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-j58tx\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.857390 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6nrlk\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.857423 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-m9qwf\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.857462 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-cts7j\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.857840 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-dockercfg-t264s\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.857977 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-nsz9w\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.857977 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.858009 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.858025 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-h468w\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.858038 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-s4fk8\"" Oct 30 18:38:50.858471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.858128 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Oct 30 18:38:50.859898 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.859312 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Oct 30 18:38:50.860176 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.860157 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-networking-console-plugin-cert\") pod \"networking-console-plugin-86c478bb88-hqzbs\" (UID: \"c135952e-12c3-4a65-bd3d-0d0b1aba49d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" Oct 30 18:38:50.860231 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.860161 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c135952e-12c3-4a65-bd3d-0d0b1aba49d1-nginx-conf\") pod \"networking-console-plugin-86c478bb88-hqzbs\" (UID: \"c135952e-12c3-4a65-bd3d-0d0b1aba49d1\") " pod="openshift-network-console/networking-console-plugin-86c478bb88-hqzbs" Oct 30 18:38:50.860231 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.860161 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/8c9b04e2-b099-4704-8786-b64a8991d0f4-original-pull-secret\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:38:50.860459 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.860442 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-cni-binary-copy\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:38:50.861145 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.861119 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-metrics-certs\") pod \"network-metrics-daemon-2s5rm\" (UID: \"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2\") " pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:38:50.861219 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.861165 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-daemon-config\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.861253 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.861222 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c89d1725-2dbb-49dc-92b1-218935ce8a74-cni-binary-copy\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.861387 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.861374 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-registry-certificates\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:50.861813 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.861796 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-installation-pull-secrets\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:50.862974 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.862957 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:38:50.863038 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.862990 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:50.863038 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.863006 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/63658d25-6d63-42e2-9755-8ca66a322d61-cert\") pod \"ingress-canary-fdrkw\" (UID: \"63658d25-6d63-42e2-9755-8ca66a322d61\") " pod="openshift-ingress-canary/ingress-canary-fdrkw" Oct 30 18:38:50.864161 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.864143 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-ovn-node-metrics-cert\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.864220 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.864149 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-ovnkube-script-lib\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.864220 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.864207 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-image-registry-private-configuration\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:50.864293 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.864277 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/4b9a31e1-73c6-4617-ac72-296e03e928f2-agent-certs\") pod \"konnectivity-agent-bchfv\" (UID: \"4b9a31e1-73c6-4617-ac72-296e03e928f2\") " pod="kube-system/konnectivity-agent-bchfv" Oct 30 18:38:50.864518 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.864503 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-trusted-ca\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:50.864971 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.864957 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7eed8239-b695-41f7-968f-b3a02659b44c-iptables-alerter-script\") pod \"iptables-alerter-lnjwb\" (UID: \"7eed8239-b695-41f7-968f-b3a02659b44c\") " pod="openshift-network-operator/iptables-alerter-lnjwb" Oct 30 18:38:50.865267 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.865243 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqsv7\" (UniqueName: \"kubernetes.io/projected/7eed8239-b695-41f7-968f-b3a02659b44c-kube-api-access-wqsv7\") pod \"iptables-alerter-lnjwb\" (UID: \"7eed8239-b695-41f7-968f-b3a02659b44c\") " pod="openshift-network-operator/iptables-alerter-lnjwb" Oct 30 18:38:50.865481 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.865460 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k44bp\" (UniqueName: \"kubernetes.io/projected/e65f9eb5-9e22-4c76-a1b5-e2250003faa6-kube-api-access-k44bp\") pod \"service-ca-operator-544d749c8c-xt7x5\" (UID: \"e65f9eb5-9e22-4c76-a1b5-e2250003faa6\") " pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" Oct 30 18:38:50.865599 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.865567 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmb9d\" (UniqueName: \"kubernetes.io/projected/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-kube-api-access-xmb9d\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.865854 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.865838 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c2tj\" (UniqueName: \"kubernetes.io/projected/9cd97653-a745-4fd6-9a6c-71aaa742b415-kube-api-access-2c2tj\") pod \"cluster-samples-operator-6ddb6c8b94-wf7rb\" (UID: \"9cd97653-a745-4fd6-9a6c-71aaa742b415\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-6ddb6c8b94-wf7rb" Oct 30 18:38:50.865926 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.865910 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l7k8\" (UniqueName: \"kubernetes.io/projected/cd195fdf-8022-4279-8eca-15d145d59038-kube-api-access-6l7k8\") pod \"network-check-target-95g7n\" (UID: \"cd195fdf-8022-4279-8eca-15d145d59038\") " pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:38:50.866750 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.866734 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-bound-sa-token\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:50.866819 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.866793 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thmjv\" (UniqueName: \"kubernetes.io/projected/6b091344-3cff-4f72-9512-02499eb9a9ef-kube-api-access-thmjv\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.866853 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.866831 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9hs4\" (UniqueName: \"kubernetes.io/projected/8d3a00e6-202e-4ffb-b8e0-67f15d831ae2-kube-api-access-b9hs4\") pod \"network-metrics-daemon-2s5rm\" (UID: \"8d3a00e6-202e-4ffb-b8e0-67f15d831ae2\") " pod="openshift-multus/network-metrics-daemon-2s5rm" Oct 30 18:38:50.866928 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.866914 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v87g2\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-kube-api-access-v87g2\") pod \"image-registry-7f6754678-55bz6\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:50.868524 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.868508 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6srz4\" (UniqueName: \"kubernetes.io/projected/7ca07043-d7b5-4660-87d4-2a92bf5dacbf-kube-api-access-6srz4\") pod \"insights-operator-7c4986d449-6kk4m\" (UID: \"7ca07043-d7b5-4660-87d4-2a92bf5dacbf\") " pod="openshift-insights/insights-operator-7c4986d449-6kk4m" Oct 30 18:38:50.868925 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.868910 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjh2m\" (UniqueName: \"kubernetes.io/projected/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-kube-api-access-qjh2m\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:38:50.869103 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.869084 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zfjf\" (UniqueName: \"kubernetes.io/projected/34a1d011-b30b-49b8-bc75-bfb9dbba3817-kube-api-access-4zfjf\") pod \"kas-connection-checker-685895c7c9-vwws2\" (UID: \"34a1d011-b30b-49b8-bc75-bfb9dbba3817\") " pod="kube-system/kas-connection-checker-685895c7c9-vwws2" Oct 30 18:38:50.869133 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.869109 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4p6x\" (UniqueName: \"kubernetes.io/projected/3180fff2-f289-47e6-a37c-34c5dff50aa2-kube-api-access-n4p6x\") pod \"router-default-797b6dc769-r2cdl\" (UID: \"3180fff2-f289-47e6-a37c-34c5dff50aa2\") " pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:38:50.869133 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.869085 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qndwx\" (UniqueName: \"kubernetes.io/projected/2bfeb3c6-d71a-4b5e-96db-198da488f7a9-kube-api-access-qndwx\") pod \"node-resolver-5mmnd\" (UID: \"2bfeb3c6-d71a-4b5e-96db-198da488f7a9\") " pod="openshift-dns/node-resolver-5mmnd" Oct 30 18:38:50.870263 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.870243 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lg2js\" (UniqueName: \"kubernetes.io/projected/2797391b-fd3d-4222-82c8-b28a2d16c9e3-kube-api-access-lg2js\") pod \"cluster-monitoring-operator-884576c57-td46q\" (UID: \"2797391b-fd3d-4222-82c8-b28a2d16c9e3\") " pod="openshift-monitoring/cluster-monitoring-operator-884576c57-td46q" Oct 30 18:38:50.870529 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.870499 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5w86\" (UniqueName: \"kubernetes.io/projected/a36f2ae5-c6ca-49cf-a775-82a97e92f2c4-kube-api-access-m5w86\") pod \"kas-connection-checker-685895c7c9-pvzsh\" (UID: \"a36f2ae5-c6ca-49cf-a775-82a97e92f2c4\") " pod="kube-system/kas-connection-checker-685895c7c9-pvzsh" Oct 30 18:38:50.870607 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.870588 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ztsmk\" (UniqueName: \"kubernetes.io/projected/e2455903-bb15-41e2-8d91-51df8775b033-kube-api-access-ztsmk\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.870856 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.870805 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6srj\" (UniqueName: \"kubernetes.io/projected/7a4adea8-c5dc-4a4d-ad54-e985838fa6c5-kube-api-access-w6srj\") pod \"kas-connection-checker-685895c7c9-mm49z\" (UID: \"7a4adea8-c5dc-4a4d-ad54-e985838fa6c5\") " pod="kube-system/kas-connection-checker-685895c7c9-mm49z" Oct 30 18:38:50.870947 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.870826 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hlr2\" (UniqueName: \"kubernetes.io/projected/63658d25-6d63-42e2-9755-8ca66a322d61-kube-api-access-7hlr2\") pod \"ingress-canary-fdrkw\" (UID: \"63658d25-6d63-42e2-9755-8ca66a322d61\") " pod="openshift-ingress-canary/ingress-canary-fdrkw" Oct 30 18:38:50.871223 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.871205 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8blzq\" (UniqueName: \"kubernetes.io/projected/89271a7f-e9c4-43dd-a320-54900b3fb8b7-kube-api-access-8blzq\") pod \"kube-storage-version-migrator-operator-7d4bc45dd6-689xp\" (UID: \"89271a7f-e9c4-43dd-a320-54900b3fb8b7\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" Oct 30 18:38:50.871299 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.871260 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fkqwc\" (UniqueName: \"kubernetes.io/projected/c89d1725-2dbb-49dc-92b1-218935ce8a74-kube-api-access-fkqwc\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.871384 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.871366 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghbwl\" (UniqueName: \"kubernetes.io/projected/67b57c01-e4a3-4ae1-ab6e-9b386e181adb-kube-api-access-ghbwl\") pod \"console-operator-7cd56b8dbc-gg799\" (UID: \"67b57c01-e4a3-4ae1-ab6e-9b386e181adb\") " pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:50.871484 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.871461 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2q9pq\" (UniqueName: \"kubernetes.io/projected/bb52d69e-0e3f-46b5-8491-a2be5a912d29-kube-api-access-2q9pq\") pod \"volume-data-source-validator-58c9f5f46d-4z4ss\" (UID: \"bb52d69e-0e3f-46b5-8491-a2be5a912d29\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-58c9f5f46d-4z4ss" Oct 30 18:38:50.871517 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.871494 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvrqw\" (UniqueName: \"kubernetes.io/projected/51d9053b-abef-45c8-bd1a-67343812dcd9-kube-api-access-lvrqw\") pod \"network-check-source-6f78d59f99-jwxvn\" (UID: \"51d9053b-abef-45c8-bd1a-67343812dcd9\") " pod="openshift-network-diagnostics/network-check-source-6f78d59f99-jwxvn" Oct 30 18:38:50.871517 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.871494 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6tt86\" (UniqueName: \"kubernetes.io/projected/15f5b642-cd06-42ad-9832-dcfe438ea4e5-kube-api-access-6tt86\") pod \"node-ca-kjn4r\" (UID: \"15f5b642-cd06-42ad-9832-dcfe438ea4e5\") " pod="openshift-image-registry/node-ca-kjn4r" Oct 30 18:38:50.915565 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915543 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.915669 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915588 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-lib-modules\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.915669 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915606 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-system-cni-dir\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.915669 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915619 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-systemd\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.915669 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915635 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7eed8239-b695-41f7-968f-b3a02659b44c-host-slash\") pod \"iptables-alerter-lnjwb\" (UID: \"7eed8239-b695-41f7-968f-b3a02659b44c\") " pod="openshift-network-operator/iptables-alerter-lnjwb" Oct 30 18:38:50.915669 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915650 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-modprobe-d\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.915865 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915666 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.915865 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915700 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-systemd\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.915865 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915706 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7eed8239-b695-41f7-968f-b3a02659b44c-host-slash\") pod \"iptables-alerter-lnjwb\" (UID: \"7eed8239-b695-41f7-968f-b3a02659b44c\") " pod="openshift-network-operator/iptables-alerter-lnjwb" Oct 30 18:38:50.915865 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915721 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-lib-modules\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.915865 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915701 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-system-cni-dir\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.915865 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915748 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-modprobe-d\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.915865 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915748 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-node-log\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.915865 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915728 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-node-log\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.915865 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915785 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh5hg\" (UniqueName: \"kubernetes.io/projected/74990028-c4d9-40d7-b1dc-b45b9f3ccbf2-kube-api-access-xh5hg\") pod \"migrator-66df5d496b-j75jm\" (UID: \"74990028-c4d9-40d7-b1dc-b45b9f3ccbf2\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j75jm" Oct 30 18:38:50.915865 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915800 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2bfeb3c6-d71a-4b5e-96db-198da488f7a9-hosts-file\") pod \"node-resolver-5mmnd\" (UID: \"2bfeb3c6-d71a-4b5e-96db-198da488f7a9\") " pod="openshift-dns/node-resolver-5mmnd" Oct 30 18:38:50.915865 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915814 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-var-lib-openvswitch\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.915865 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915829 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8c9b04e2-b099-4704-8786-b64a8991d0f4-dbus\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:38:50.915865 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915846 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-device-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915872 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2bfeb3c6-d71a-4b5e-96db-198da488f7a9-hosts-file\") pod \"node-resolver-5mmnd\" (UID: \"2bfeb3c6-d71a-4b5e-96db-198da488f7a9\") " pod="openshift-dns/node-resolver-5mmnd" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915884 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-device-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915883 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-var-lib-openvswitch\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915896 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/8c9b04e2-b099-4704-8786-b64a8991d0f4-dbus\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915902 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-run-k8s-cni-cncf-io\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915933 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-run-netns\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915951 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-run-k8s-cni-cncf-io\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915962 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915981 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-run-netns\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.915989 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916014 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca-signing-key\") pod \"service-ca-5fffc97bdf-t2jzr\" (UID: \"77f8df88-fa80-42a5-9ac6-33b2fea5d7ca\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916037 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca-signing-cabundle\") pod \"service-ca-5fffc97bdf-t2jzr\" (UID: \"77f8df88-fa80-42a5-9ac6-33b2fea5d7ca\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916072 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-socket-dir-parent\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916089 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-hostroot\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916104 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-var-lib-kubelet\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916112 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916128 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-run-ovn\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916154 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-socket-dir-parent\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916156 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-hostroot\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916161 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-var-lib-kubelet\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916152 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbjt2\" (UniqueName: \"kubernetes.io/projected/91b2497b-8770-4c64-9b1f-e2ec8744ccf9-kube-api-access-cbjt2\") pod \"migrator-66df5d496b-fjdk5\" (UID: \"91b2497b-8770-4c64-9b1f-e2ec8744ccf9\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fjdk5" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916224 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3d5e33b3-69cd-4fc7-8366-14ffa038713e-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916235 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca-signing-cabundle\") pod \"service-ca-5fffc97bdf-t2jzr\" (UID: \"77f8df88-fa80-42a5-9ac6-33b2fea5d7ca\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916189 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-run-ovn\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916238 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-cni-dir\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916278 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-cni-dir\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916285 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-sysctl-conf\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916317 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-run-systemd\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916343 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v72l5\" (UniqueName: \"kubernetes.io/projected/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-kube-api-access-v72l5\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916359 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-sysctl-conf\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916366 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-run-systemd\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916345 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca-signing-key\") pod \"service-ca-5fffc97bdf-t2jzr\" (UID: \"77f8df88-fa80-42a5-9ac6-33b2fea5d7ca\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916368 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8c9b04e2-b099-4704-8786-b64a8991d0f4-kubelet-config\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:38:50.916405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916406 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-kubernetes\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916432 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-systemd-units\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916410 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/8c9b04e2-b099-4704-8786-b64a8991d0f4-kubelet-config\") pod \"global-pull-secret-syncer-hmzzs\" (UID: \"8c9b04e2-b099-4704-8786-b64a8991d0f4\") " pod="kube-system/global-pull-secret-syncer-hmzzs" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916451 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-kubernetes\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916455 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-slash\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916475 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-systemd-units\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916477 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-cni-netd\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916514 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-slash\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916522 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916506 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-cni-netd\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916548 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916554 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d5e33b3-69cd-4fc7-8366-14ffa038713e-data-volume\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916596 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-sysctl-d\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916622 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916630 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3d5e33b3-69cd-4fc7-8366-14ffa038713e-data-volume\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916644 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15f5b642-cd06-42ad-9832-dcfe438ea4e5-host\") pod \"node-ca-kjn4r\" (UID: \"15f5b642-cd06-42ad-9832-dcfe438ea4e5\") " pod="openshift-image-registry/node-ca-kjn4r" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916686 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-sysctl-d\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916687 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/15f5b642-cd06-42ad-9832-dcfe438ea4e5-host\") pod \"node-ca-kjn4r\" (UID: \"15f5b642-cd06-42ad-9832-dcfe438ea4e5\") " pod="openshift-image-registry/node-ca-kjn4r" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916734 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-run-openvswitch\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916765 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-var-lib-kubelet\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916778 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-run-openvswitch\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916791 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-etc-selinux\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916803 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-var-lib-kubelet\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916809 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-metrics-tls\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916818 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-run\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916837 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-host\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916851 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-sys-fs\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916865 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwgvn\" (UniqueName: \"kubernetes.io/projected/290aeefa-831a-4e2d-bc4c-11defdfee2d0-kube-api-access-qwgvn\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916870 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-etc-selinux\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916882 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6f6l\" (UniqueName: \"kubernetes.io/projected/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca-kube-api-access-b6f6l\") pod \"service-ca-5fffc97bdf-t2jzr\" (UID: \"77f8df88-fa80-42a5-9ac6-33b2fea5d7ca\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916895 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-host\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916926 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-sys-fs\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916972 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-sys\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916992 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-service-ca-bundle\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.916994 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-run\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917010 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3d5e33b3-69cd-4fc7-8366-14ffa038713e-crio-socket\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917025 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-etc-kubernetes\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917042 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-var-lib-cni-multus\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917040 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-sys\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917085 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-etc-kubernetes\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917089 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-conf-dir\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917114 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-var-lib-cni-multus\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917125 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-etc-openvswitch\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917139 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-multus-conf-dir\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917154 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-socket-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917178 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-registration-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917184 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-etc-openvswitch\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917203 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/290aeefa-831a-4e2d-bc4c-11defdfee2d0-archives-path\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917214 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-registration-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917202 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e2455903-bb15-41e2-8d91-51df8775b033-socket-dir\") pod \"aws-ebs-csi-driver-node-hdd4v\" (UID: \"e2455903-bb15-41e2-8d91-51df8775b033\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hdd4v" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917231 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-run-netns\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917256 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-sysconfig\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917265 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/290aeefa-831a-4e2d-bc4c-11defdfee2d0-archives-path\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917283 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-log-socket\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917299 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/6b091344-3cff-4f72-9512-02499eb9a9ef-etc-sysconfig\") pod \"tuned-mc5j7\" (UID: \"6b091344-3cff-4f72-9512-02499eb9a9ef\") " pod="openshift-cluster-node-tuning-operator/tuned-mc5j7" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917304 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-run-netns\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917306 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-run-ovn-kubernetes\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917340 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-run-ovn-kubernetes\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917329 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-log-socket\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917363 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-cnibin\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917397 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4w8p5\" (UniqueName: \"kubernetes.io/projected/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-api-access-4w8p5\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917412 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-cnibin\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917423 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-config-volume\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917449 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-system-cni-dir\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917479 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-var-lib-cni-bin\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917499 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-system-cni-dir\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917502 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-cni-bin\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917526 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-var-lib-cni-bin\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917527 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917533 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-service-ca-bundle\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917552 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-cni-bin\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917555 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-trusted-ca-bundle\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917594 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-run-multus-certs\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917612 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-tmp-dir\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917612 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00-tuning-conf-dir\") pod \"multus-additional-cni-plugins-bh8c5\" (UID: \"44c58e2f-bf8e-41d1-9bb2-be8fc0e2eb00\") " pod="openshift-multus/multus-additional-cni-plugins-bh8c5" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917638 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-config-volume\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917650 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-host-run-multus-certs\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917677 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-cnibin\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917693 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-kubelet\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917698 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-tmp-dir\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917723 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ce6f0c54-70e4-4d45-8bc3-2823a815f0db-host-kubelet\") pod \"ovnkube-node-ls845\" (UID: \"ce6f0c54-70e4-4d45-8bc3-2823a815f0db\") " pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917742 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c89d1725-2dbb-49dc-92b1-218935ce8a74-cnibin\") pod \"multus-m5xnp\" (UID: \"c89d1725-2dbb-49dc-92b1-218935ce8a74\") " pod="openshift-multus/multus-m5xnp" Oct 30 18:38:50.917805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.917815 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-trusted-ca-bundle\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:50.970847 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.970822 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh5hg\" (UniqueName: \"kubernetes.io/projected/74990028-c4d9-40d7-b1dc-b45b9f3ccbf2-kube-api-access-xh5hg\") pod \"migrator-66df5d496b-j75jm\" (UID: \"74990028-c4d9-40d7-b1dc-b45b9f3ccbf2\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-j75jm" Oct 30 18:38:50.970936 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.970822 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v72l5\" (UniqueName: \"kubernetes.io/projected/a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f-kube-api-access-v72l5\") pod \"dns-default-kkzsh\" (UID: \"a2195bb6-ac20-4e5f-b3e9-69b6e5b9b98f\") " pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:50.970936 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.970869 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbjt2\" (UniqueName: \"kubernetes.io/projected/91b2497b-8770-4c64-9b1f-e2ec8744ccf9-kube-api-access-cbjt2\") pod \"migrator-66df5d496b-fjdk5\" (UID: \"91b2497b-8770-4c64-9b1f-e2ec8744ccf9\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-fjdk5" Oct 30 18:38:50.971256 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.971240 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4w8p5\" (UniqueName: \"kubernetes.io/projected/3d5e33b3-69cd-4fc7-8366-14ffa038713e-kube-api-access-4w8p5\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:50.971305 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.971286 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6f6l\" (UniqueName: \"kubernetes.io/projected/77f8df88-fa80-42a5-9ac6-33b2fea5d7ca-kube-api-access-b6f6l\") pod \"service-ca-5fffc97bdf-t2jzr\" (UID: \"77f8df88-fa80-42a5-9ac6-33b2fea5d7ca\") " pod="openshift-service-ca/service-ca-5fffc97bdf-t2jzr" Oct 30 18:38:50.971387 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:50.971373 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwgvn\" (UniqueName: \"kubernetes.io/projected/290aeefa-831a-4e2d-bc4c-11defdfee2d0-kube-api-access-qwgvn\") pod \"periodic-gathering-jmjn6-ghcr6\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:38:51.018804 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:51.018767 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3d5e33b3-69cd-4fc7-8366-14ffa038713e-crio-socket\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:51.018920 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:51.018878 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3d5e33b3-69cd-4fc7-8366-14ffa038713e-crio-socket\") pod \"insights-runtime-extractor-5bn4v\" (UID: \"3d5e33b3-69cd-4fc7-8366-14ffa038713e\") " pod="openshift-insights/insights-runtime-extractor-5bn4v" Oct 30 18:38:51.460548 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:51.460518 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:51.532020 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:51.531981 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:51.627061 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:51.627034 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-95g7n" Oct 30 18:38:51.719642 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:51.719555 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:51.951070 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:51.951039 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:52.180076 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.180047 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:52.362455 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.362419 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:52.472163 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.472094 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-k4tjq"] Oct 30 18:38:52.494716 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.494686 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.531690 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.531660 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b46058e-2df9-4fa1-bd75-7d24f1229256-sys\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.531834 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.531705 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7b46058e-2df9-4fa1-bd75-7d24f1229256-root\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.531834 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.531721 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b46058e-2df9-4fa1-bd75-7d24f1229256-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.531834 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.531798 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b46058e-2df9-4fa1-bd75-7d24f1229256-metrics-client-ca\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.531951 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.531885 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7b46058e-2df9-4fa1-bd75-7d24f1229256-node-exporter-textfile\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.531951 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.531914 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b46058e-2df9-4fa1-bd75-7d24f1229256-node-exporter-tls\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.531951 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.531934 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7b46058e-2df9-4fa1-bd75-7d24f1229256-node-exporter-accelerators-collector-config\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.532055 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.531951 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cb67v\" (UniqueName: \"kubernetes.io/projected/7b46058e-2df9-4fa1-bd75-7d24f1229256-kube-api-access-cb67v\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.532089 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.532057 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7b46058e-2df9-4fa1-bd75-7d24f1229256-node-exporter-wtmp\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.540374 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.540349 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:52.548930 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.548909 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Oct 30 18:38:52.549164 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.549149 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Oct 30 18:38:52.549318 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.549295 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-xqkxc\"" Oct 30 18:38:52.630815 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.630786 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Oct 30 18:38:52.631310 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.630908 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Oct 30 18:38:52.633190 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.633162 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b46058e-2df9-4fa1-bd75-7d24f1229256-sys\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.633330 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.633217 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7b46058e-2df9-4fa1-bd75-7d24f1229256-root\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.633330 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.633251 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b46058e-2df9-4fa1-bd75-7d24f1229256-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.633330 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.633277 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7b46058e-2df9-4fa1-bd75-7d24f1229256-sys\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.633330 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.633285 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b46058e-2df9-4fa1-bd75-7d24f1229256-metrics-client-ca\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.633542 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.633344 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7b46058e-2df9-4fa1-bd75-7d24f1229256-node-exporter-textfile\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.633542 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.633378 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b46058e-2df9-4fa1-bd75-7d24f1229256-node-exporter-tls\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.633542 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.633415 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7b46058e-2df9-4fa1-bd75-7d24f1229256-node-exporter-accelerators-collector-config\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.633542 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.633442 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cb67v\" (UniqueName: \"kubernetes.io/projected/7b46058e-2df9-4fa1-bd75-7d24f1229256-kube-api-access-cb67v\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.633542 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.633345 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/7b46058e-2df9-4fa1-bd75-7d24f1229256-root\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.633542 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.633469 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7b46058e-2df9-4fa1-bd75-7d24f1229256-node-exporter-wtmp\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.633845 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.633636 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/7b46058e-2df9-4fa1-bd75-7d24f1229256-node-exporter-wtmp\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.633950 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.633928 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7b46058e-2df9-4fa1-bd75-7d24f1229256-metrics-client-ca\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.634016 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.634001 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/7b46058e-2df9-4fa1-bd75-7d24f1229256-node-exporter-accelerators-collector-config\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.634233 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.634213 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/7b46058e-2df9-4fa1-bd75-7d24f1229256-node-exporter-textfile\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.634303 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.634240 5894 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Oct 30 18:38:52.636391 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.636364 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7b46058e-2df9-4fa1-bd75-7d24f1229256-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.636924 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.636902 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/7b46058e-2df9-4fa1-bd75-7d24f1229256-node-exporter-tls\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.689635 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.689600 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cb67v\" (UniqueName: \"kubernetes.io/projected/7b46058e-2df9-4fa1-bd75-7d24f1229256-kube-api-access-cb67v\") pod \"node-exporter-k4tjq\" (UID: \"7b46058e-2df9-4fa1-bd75-7d24f1229256\") " pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.730005 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.729933 5894 generic.go:356] "Generic (PLEG): container finished" podID="e65f9eb5-9e22-4c76-a1b5-e2250003faa6" containerID="b8f824433003d6bed1a3f3b7317e82b3ca4dad9a02be9fbe9e6e1e510d9be232" exitCode=0 Oct 30 18:38:52.730128 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.730005 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" event={"ID":"e65f9eb5-9e22-4c76-a1b5-e2250003faa6","Type":"ContainerDied","Data":"b8f824433003d6bed1a3f3b7317e82b3ca4dad9a02be9fbe9e6e1e510d9be232"} Oct 30 18:38:52.730342 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.730328 5894 scope.go:122] "RemoveContainer" containerID="b8f824433003d6bed1a3f3b7317e82b3ca4dad9a02be9fbe9e6e1e510d9be232" Oct 30 18:38:52.731976 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.731956 5894 generic.go:356] "Generic (PLEG): container finished" podID="89271a7f-e9c4-43dd-a320-54900b3fb8b7" containerID="cbf5a74d4fe9a24e94f0a15f52455e21ede7e50440a9df950ebd217e34bb0b7e" exitCode=0 Oct 30 18:38:52.732085 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.732057 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" event={"ID":"89271a7f-e9c4-43dd-a320-54900b3fb8b7","Type":"ContainerDied","Data":"cbf5a74d4fe9a24e94f0a15f52455e21ede7e50440a9df950ebd217e34bb0b7e"} Oct 30 18:38:52.732365 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.732338 5894 scope.go:122] "RemoveContainer" containerID="cbf5a74d4fe9a24e94f0a15f52455e21ede7e50440a9df950ebd217e34bb0b7e" Oct 30 18:38:52.733783 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.733767 5894 generic.go:356] "Generic (PLEG): container finished" podID="7ca07043-d7b5-4660-87d4-2a92bf5dacbf" containerID="144cbb89729ed1c73163dfde2cbe9d352e2d6f35c4a8dd0fd0fe6a7e0cafd3bf" exitCode=0 Oct 30 18:38:52.733854 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.733816 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-6kk4m" event={"ID":"7ca07043-d7b5-4660-87d4-2a92bf5dacbf","Type":"ContainerDied","Data":"144cbb89729ed1c73163dfde2cbe9d352e2d6f35c4a8dd0fd0fe6a7e0cafd3bf"} Oct 30 18:38:52.734073 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.734054 5894 scope.go:122] "RemoveContainer" containerID="144cbb89729ed1c73163dfde2cbe9d352e2d6f35c4a8dd0fd0fe6a7e0cafd3bf" Oct 30 18:38:52.847409 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.847382 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k4tjq" Oct 30 18:38:52.873947 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:38:52.873918 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b46058e_2df9_4fa1_bd75_7d24f1229256.slice/crio-e9f63ab17a96811bf4f6f4cdb4ba093205146d70212a38a5990e03349c0cf473 WatchSource:0}: Error finding container e9f63ab17a96811bf4f6f4cdb4ba093205146d70212a38a5990e03349c0cf473: Status 404 returned error can't find the container with id e9f63ab17a96811bf4f6f4cdb4ba093205146d70212a38a5990e03349c0cf473 Oct 30 18:38:52.878516 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.878500 5894 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:38:52.994321 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:52.994244 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:53.178285 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:53.178223 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:53.353693 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:53.353618 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:53.733727 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:53.733691 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:38:53.741401 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:53.741153 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7cd56b8dbc-gg799" Oct 30 18:38:53.743779 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:53.743745 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-7c4986d449-6kk4m" event={"ID":"7ca07043-d7b5-4660-87d4-2a92bf5dacbf","Type":"ContainerStarted","Data":"4b284c6c934107502b54be1961d0d66512928615965568e156078c3111143de6"} Oct 30 18:38:53.746215 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:53.746187 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-544d749c8c-xt7x5" event={"ID":"e65f9eb5-9e22-4c76-a1b5-e2250003faa6","Type":"ContainerStarted","Data":"79a8ea9b15b83e2f26b104d02a6fde37e59182f8ab81e548d2b3a67bb4aaf487"} Oct 30 18:38:53.748635 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:53.748610 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7d4bc45dd6-689xp" event={"ID":"89271a7f-e9c4-43dd-a320-54900b3fb8b7","Type":"ContainerStarted","Data":"6662dd3af28ebd0664b31ceef16baecccc80717d8a09bc454e44e41639787ef4"} Oct 30 18:38:53.750484 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:53.750455 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k4tjq" event={"ID":"7b46058e-2df9-4fa1-bd75-7d24f1229256","Type":"ContainerStarted","Data":"e9f63ab17a96811bf4f6f4cdb4ba093205146d70212a38a5990e03349c0cf473"} Oct 30 18:38:53.836474 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:53.836436 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:54.027479 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:54.027403 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:54.447830 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:54.447793 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:54.622899 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:54.622866 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:54.871587 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:54.871547 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:55.760223 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:55.760185 5894 generic.go:356] "Generic (PLEG): container finished" podID="7b46058e-2df9-4fa1-bd75-7d24f1229256" containerID="d726060c335923f941d11bcc725539737fc84994dcc3781b52870ab254ab28e1" exitCode=0 Oct 30 18:38:55.760389 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:55.760273 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k4tjq" event={"ID":"7b46058e-2df9-4fa1-bd75-7d24f1229256","Type":"ContainerDied","Data":"d726060c335923f941d11bcc725539737fc84994dcc3781b52870ab254ab28e1"} Oct 30 18:38:56.091936 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:56.091864 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:56.497051 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:56.497019 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:56.664352 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:56.664319 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:56.767706 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:56.767380 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k4tjq" event={"ID":"7b46058e-2df9-4fa1-bd75-7d24f1229256","Type":"ContainerStarted","Data":"db95fdffe2401fc0f78cfb6f551fd022f7161192618c307e0d46008902a003c2"} Oct 30 18:38:56.767706 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:56.767444 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k4tjq" event={"ID":"7b46058e-2df9-4fa1-bd75-7d24f1229256","Type":"ContainerStarted","Data":"49e8691f76b717b0a055e991d037630bee2e7874689a86d65a888335bae90e33"} Oct 30 18:38:56.830728 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:56.830698 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:56.882599 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:56.882529 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-k4tjq" podStartSLOduration=2.88672573 podStartE2EDuration="4.882513674s" totalImagesPullingTime="1.917022601s" totalInitContainerRuntime="78.765343ms" isStatefulPod=true podCreationTimestamp="2026-10-30 18:38:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:38:56.8821804 +0000 UTC m=+7.372308221" watchObservedRunningTime="2026-10-30 18:38:56.882513674 +0000 UTC m=+7.372641487" Oct 30 18:38:57.010660 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:57.010628 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:57.181344 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:57.181313 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:57.347336 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:57.347306 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:57.727869 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:57.727842 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-kkzsh" Oct 30 18:38:58.208050 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:58.208020 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:58.384010 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:58.383966 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:38:58.670107 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:58.670081 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:38:58.670295 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:58.670281 5894 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Oct 30 18:38:59.476982 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:38:59.476954 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-797b6dc769-r2cdl" Oct 30 18:39:00.882347 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:00.882317 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-ls845" Oct 30 18:39:01.369685 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:01.369650 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:02.675746 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:02.675718 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:03.548156 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:03.548120 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f6754678-55bz6"] Oct 30 18:39:03.552500 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:03.552472 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:39:03.637863 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:03.637830 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:03.807605 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:03.807521 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:03.978772 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:03.978741 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:04.209827 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:04.209794 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:07.358821 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:07.358787 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:07.526014 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:07.525983 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:07.929222 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:07.929189 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:08.098755 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:08.098723 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:09.384853 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:09.384815 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:09.557994 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:09.557963 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:09.729253 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:09.729225 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:10.091264 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:10.091182 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:10.260901 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:10.260868 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:11.534879 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:11.534843 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:11.731161 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:11.731122 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:11.948490 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:11.948461 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:12.298140 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:12.298068 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bchfv" Oct 30 18:39:13.195461 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.195427 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-jmjn6-ghcr6"] Oct 30 18:39:13.196050 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.195677 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" podUID="290aeefa-831a-4e2d-bc4c-11defdfee2d0" containerName="insights-gathering" containerID="cri-o://a7014b135fd44083e8acfe64076a984ac9caf501ddd92d5875d7923cebd3023d" gracePeriod=30 Oct 30 18:39:13.436743 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.436720 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:39:13.504812 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.504747 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/290aeefa-831a-4e2d-bc4c-11defdfee2d0-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/290aeefa-831a-4e2d-bc4c-11defdfee2d0-archives-path\") pod \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " Oct 30 18:39:13.504812 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.504790 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-service-ca-bundle\") pod \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " Oct 30 18:39:13.504959 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.504864 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-trusted-ca-bundle\") pod \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " Oct 30 18:39:13.504959 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.504883 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/290aeefa-831a-4e2d-bc4c-11defdfee2d0-kube-api-access-qwgvn\" (UniqueName: \"kubernetes.io/projected/290aeefa-831a-4e2d-bc4c-11defdfee2d0-kube-api-access-qwgvn\") pod \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\" (UID: \"290aeefa-831a-4e2d-bc4c-11defdfee2d0\") " Oct 30 18:39:13.505127 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.505102 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/290aeefa-831a-4e2d-bc4c-11defdfee2d0-archives-path" pod "290aeefa-831a-4e2d-bc4c-11defdfee2d0" (UID: "290aeefa-831a-4e2d-bc4c-11defdfee2d0"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:39:13.505211 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.505192 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-trusted-ca-bundle" pod "290aeefa-831a-4e2d-bc4c-11defdfee2d0" (UID: "290aeefa-831a-4e2d-bc4c-11defdfee2d0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:39:13.505248 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.505216 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-service-ca-bundle" pod "290aeefa-831a-4e2d-bc4c-11defdfee2d0" (UID: "290aeefa-831a-4e2d-bc4c-11defdfee2d0"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:39:13.507094 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.507051 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/290aeefa-831a-4e2d-bc4c-11defdfee2d0-kube-api-access-qwgvn" pod "290aeefa-831a-4e2d-bc4c-11defdfee2d0" (UID: "290aeefa-831a-4e2d-bc4c-11defdfee2d0"). InnerVolumeSpecName "kube-api-access-qwgvn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:39:13.605596 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.605547 5894 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-trusted-ca-bundle\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:39:13.605596 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.605597 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwgvn\" (UniqueName: \"kubernetes.io/projected/290aeefa-831a-4e2d-bc4c-11defdfee2d0-kube-api-access-qwgvn\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:39:13.605784 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.605610 5894 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/290aeefa-831a-4e2d-bc4c-11defdfee2d0-archives-path\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:39:13.605784 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.605620 5894 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/290aeefa-831a-4e2d-bc4c-11defdfee2d0-service-ca-bundle\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:39:13.831249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.831170 5894 generic.go:356] "Generic (PLEG): container finished" podID="290aeefa-831a-4e2d-bc4c-11defdfee2d0" containerID="a7014b135fd44083e8acfe64076a984ac9caf501ddd92d5875d7923cebd3023d" exitCode=2 Oct 30 18:39:13.831249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.831218 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" Oct 30 18:39:13.831414 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.831256 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" event={"ID":"290aeefa-831a-4e2d-bc4c-11defdfee2d0","Type":"ContainerDied","Data":"a7014b135fd44083e8acfe64076a984ac9caf501ddd92d5875d7923cebd3023d"} Oct 30 18:39:13.831414 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.831315 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-jmjn6-ghcr6" event={"ID":"290aeefa-831a-4e2d-bc4c-11defdfee2d0","Type":"ContainerDied","Data":"ff0ec68a90647ce926cdb4307ff2be59548cfbdc73a2e046c61eade4003464e8"} Oct 30 18:39:13.831414 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.831338 5894 scope.go:122] "RemoveContainer" containerID="a7014b135fd44083e8acfe64076a984ac9caf501ddd92d5875d7923cebd3023d" Oct 30 18:39:13.842393 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.842374 5894 scope.go:122] "RemoveContainer" containerID="a7014b135fd44083e8acfe64076a984ac9caf501ddd92d5875d7923cebd3023d" Oct 30 18:39:13.842820 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:39:13.842799 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7014b135fd44083e8acfe64076a984ac9caf501ddd92d5875d7923cebd3023d\": container with ID starting with a7014b135fd44083e8acfe64076a984ac9caf501ddd92d5875d7923cebd3023d not found: ID does not exist" containerID="a7014b135fd44083e8acfe64076a984ac9caf501ddd92d5875d7923cebd3023d" Oct 30 18:39:13.842899 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.842833 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7014b135fd44083e8acfe64076a984ac9caf501ddd92d5875d7923cebd3023d"} err="failed to get container status \"a7014b135fd44083e8acfe64076a984ac9caf501ddd92d5875d7923cebd3023d\": rpc error: code = NotFound desc = could not find container \"a7014b135fd44083e8acfe64076a984ac9caf501ddd92d5875d7923cebd3023d\": container with ID starting with a7014b135fd44083e8acfe64076a984ac9caf501ddd92d5875d7923cebd3023d not found: ID does not exist" Oct 30 18:39:13.992623 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:13.992591 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-insights/periodic-gathering-jmjn6-ghcr6"] Oct 30 18:39:14.042101 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:14.042071 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-insights/periodic-gathering-jmjn6-ghcr6"] Oct 30 18:39:15.647389 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:15.647353 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="290aeefa-831a-4e2d-bc4c-11defdfee2d0" path="/var/lib/kubelet/pods/290aeefa-831a-4e2d-bc4c-11defdfee2d0/volumes" Oct 30 18:39:16.531095 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:16.531058 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:20.117328 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:39:20.117279 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:39:22.329421 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:22.329393 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:22.505909 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:22.505879 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:27.580792 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:27.580758 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:27.752963 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:27.752925 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:28.567853 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.567811 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7f6754678-55bz6" podUID="4a88dde4-2103-476d-81c4-4b255cff4828" containerName="registry" containerID="cri-o://1f208f81f72a5c903471774bf4293139af22fff813196ac6c473474aebec507d" gracePeriod=30 Oct 30 18:39:28.813252 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.813228 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:39:28.892150 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.892124 5894 generic.go:356] "Generic (PLEG): container finished" podID="4a88dde4-2103-476d-81c4-4b255cff4828" containerID="1f208f81f72a5c903471774bf4293139af22fff813196ac6c473474aebec507d" exitCode=0 Oct 30 18:39:28.892293 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.892165 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f6754678-55bz6" Oct 30 18:39:28.892293 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.892206 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f6754678-55bz6" event={"ID":"4a88dde4-2103-476d-81c4-4b255cff4828","Type":"ContainerDied","Data":"1f208f81f72a5c903471774bf4293139af22fff813196ac6c473474aebec507d"} Oct 30 18:39:28.892293 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.892265 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f6754678-55bz6" event={"ID":"4a88dde4-2103-476d-81c4-4b255cff4828","Type":"ContainerDied","Data":"4971600d17965249e566952e3d5de9e09491c9d11640b0f63933dc353ea1ab09"} Oct 30 18:39:28.892293 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.892286 5894 scope.go:122] "RemoveContainer" containerID="1f208f81f72a5c903471774bf4293139af22fff813196ac6c473474aebec507d" Oct 30 18:39:28.900100 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.900079 5894 scope.go:122] "RemoveContainer" containerID="1f208f81f72a5c903471774bf4293139af22fff813196ac6c473474aebec507d" Oct 30 18:39:28.900340 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:39:28.900321 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f208f81f72a5c903471774bf4293139af22fff813196ac6c473474aebec507d\": container with ID starting with 1f208f81f72a5c903471774bf4293139af22fff813196ac6c473474aebec507d not found: ID does not exist" containerID="1f208f81f72a5c903471774bf4293139af22fff813196ac6c473474aebec507d" Oct 30 18:39:28.900393 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.900351 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f208f81f72a5c903471774bf4293139af22fff813196ac6c473474aebec507d"} err="failed to get container status \"1f208f81f72a5c903471774bf4293139af22fff813196ac6c473474aebec507d\": rpc error: code = NotFound desc = could not find container \"1f208f81f72a5c903471774bf4293139af22fff813196ac6c473474aebec507d\": container with ID starting with 1f208f81f72a5c903471774bf4293139af22fff813196ac6c473474aebec507d not found: ID does not exist" Oct 30 18:39:28.928068 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.928049 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-bound-sa-token\") pod \"4a88dde4-2103-476d-81c4-4b255cff4828\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " Oct 30 18:39:28.928157 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.928078 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-trusted-ca\") pod \"4a88dde4-2103-476d-81c4-4b255cff4828\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " Oct 30 18:39:28.928157 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.928109 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-kube-api-access-v87g2\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-kube-api-access-v87g2\") pod \"4a88dde4-2103-476d-81c4-4b255cff4828\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " Oct 30 18:39:28.928157 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.928133 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls\") pod \"4a88dde4-2103-476d-81c4-4b255cff4828\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " Oct 30 18:39:28.928321 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.928189 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-registry-certificates\") pod \"4a88dde4-2103-476d-81c4-4b255cff4828\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " Oct 30 18:39:28.928321 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.928211 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-image-registry-private-configuration\") pod \"4a88dde4-2103-476d-81c4-4b255cff4828\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " Oct 30 18:39:28.928321 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.928253 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4a88dde4-2103-476d-81c4-4b255cff4828-ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a88dde4-2103-476d-81c4-4b255cff4828-ca-trust-extracted\") pod \"4a88dde4-2103-476d-81c4-4b255cff4828\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " Oct 30 18:39:28.928321 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.928291 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-installation-pull-secrets\") pod \"4a88dde4-2103-476d-81c4-4b255cff4828\" (UID: \"4a88dde4-2103-476d-81c4-4b255cff4828\") " Oct 30 18:39:28.928667 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.928617 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-trusted-ca" pod "4a88dde4-2103-476d-81c4-4b255cff4828" (UID: "4a88dde4-2103-476d-81c4-4b255cff4828"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:39:28.928875 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.928820 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-registry-certificates" pod "4a88dde4-2103-476d-81c4-4b255cff4828" (UID: "4a88dde4-2103-476d-81c4-4b255cff4828"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:39:28.930718 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.930686 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-installation-pull-secrets" pod "4a88dde4-2103-476d-81c4-4b255cff4828" (UID: "4a88dde4-2103-476d-81c4-4b255cff4828"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:39:28.930830 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.930802 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls" pod "4a88dde4-2103-476d-81c4-4b255cff4828" (UID: "4a88dde4-2103-476d-81c4-4b255cff4828"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:39:28.930917 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.930889 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-bound-sa-token" pod "4a88dde4-2103-476d-81c4-4b255cff4828" (UID: "4a88dde4-2103-476d-81c4-4b255cff4828"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:39:28.931002 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.930922 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-image-registry-private-configuration" pod "4a88dde4-2103-476d-81c4-4b255cff4828" (UID: "4a88dde4-2103-476d-81c4-4b255cff4828"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:39:28.931002 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.930964 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-kube-api-access-v87g2" pod "4a88dde4-2103-476d-81c4-4b255cff4828" (UID: "4a88dde4-2103-476d-81c4-4b255cff4828"). InnerVolumeSpecName "kube-api-access-v87g2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:39:28.935166 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:28.935133 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a88dde4-2103-476d-81c4-4b255cff4828-ca-trust-extracted" pod "4a88dde4-2103-476d-81c4-4b255cff4828" (UID: "4a88dde4-2103-476d-81c4-4b255cff4828"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:39:29.029353 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:29.029319 5894 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4a88dde4-2103-476d-81c4-4b255cff4828-ca-trust-extracted\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:39:29.029353 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:29.029352 5894 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-installation-pull-secrets\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:39:29.029791 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:29.029761 5894 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-bound-sa-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:39:29.029791 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:29.029791 5894 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-trusted-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:39:29.029946 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:29.029806 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v87g2\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-kube-api-access-v87g2\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:39:29.029946 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:29.029819 5894 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4a88dde4-2103-476d-81c4-4b255cff4828-registry-tls\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:39:29.029946 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:29.029832 5894 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4a88dde4-2103-476d-81c4-4b255cff4828-registry-certificates\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:39:29.029946 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:29.029847 5894 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4a88dde4-2103-476d-81c4-4b255cff4828-image-registry-private-configuration\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:39:29.310064 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:29.310031 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f6754678-55bz6"] Oct 30 18:39:29.314138 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:29.314117 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7f6754678-55bz6"] Oct 30 18:39:29.648261 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:29.648234 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4a88dde4-2103-476d-81c4-4b255cff4828" path="/var/lib/kubelet/pods/4a88dde4-2103-476d-81c4-4b255cff4828/volumes" Oct 30 18:39:36.347265 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:36.347229 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:36.521075 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:36.521039 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:37.323904 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:37.323867 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:37.494992 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:37.494952 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:44.088141 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:44.088105 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:48.594166 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:48.594131 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:48.767325 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:48.767290 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:48.979453 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:48.979395 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:49.153809 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:49.153781 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:50.116729 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:39:50.116690 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:39:52.321347 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:52.321316 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:52.541079 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:52.541047 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:58.700603 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:58.700558 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:39:58.877467 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:39:58.877438 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:07.328546 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:40:07.328488 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:07.501824 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:40:07.501784 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:12.042130 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:40:12.042094 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:20.116149 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:40:20.116114 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:40:21.295354 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:40:21.295319 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:41.800147 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:40:41.800070 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:40:50.116668 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:40:50.116634 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:41:12.401357 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:41:12.401321 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:41:20.116093 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:41:20.116057 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:41:50.116782 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:41:50.116749 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:42:09.708486 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:09.708411 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:09.912298 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:09.912262 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:11.751852 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:11.751816 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:11.931405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:11.931371 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:12.717785 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:12.717754 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:14.703673 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:14.703641 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:14.866999 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:14.866964 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:20.116924 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:42:20.116889 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:42:36.419116 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:36.419082 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:37.203245 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:37.203209 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:37.382258 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:37.382209 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:39.626703 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:39.626664 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:39.804753 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:39.804684 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:42.914690 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:42.914658 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:43.121449 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:43.121418 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:43.601958 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:43.601926 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:43.825128 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:43.825099 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:45.355665 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:45.355632 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:46.016679 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:46.016640 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:46.193885 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:46.193851 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:46.357037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:46.356966 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:50.116621 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:42:50.116583 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:42:50.187023 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:50.186994 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:50.363619 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:50.363564 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:54.281313 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:54.281273 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:54.450857 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:54.450823 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:54.748802 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:54.748771 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:42:55.915916 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:55.915883 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-597b95cb84-h9xf2"] Oct 30 18:42:55.916646 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:55.916623 5894 cpu_manager.go:414] "containerMap: removing container" podUID="290aeefa-831a-4e2d-bc4c-11defdfee2d0" containerName="insights-gathering" Oct 30 18:42:55.916646 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:55.916641 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="290aeefa-831a-4e2d-bc4c-11defdfee2d0" containerName="insights-gathering" Oct 30 18:42:55.916797 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:55.916658 5894 cpu_manager.go:414] "containerMap: removing container" podUID="4a88dde4-2103-476d-81c4-4b255cff4828" containerName="registry" Oct 30 18:42:55.916797 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:55.916663 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4a88dde4-2103-476d-81c4-4b255cff4828" containerName="registry" Oct 30 18:42:55.916797 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:55.916724 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="290aeefa-831a-4e2d-bc4c-11defdfee2d0" containerName="insights-gathering" Oct 30 18:42:55.916797 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:55.916733 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="4a88dde4-2103-476d-81c4-4b255cff4828" containerName="registry" Oct 30 18:42:55.919447 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:55.919431 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-h9xf2" Oct 30 18:42:55.971833 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:55.971810 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-r7zbx\"" Oct 30 18:42:55.971945 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:55.971868 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Oct 30 18:42:55.971945 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:55.971905 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Oct 30 18:42:56.022039 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:56.022006 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-h9xf2"] Oct 30 18:42:56.025742 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:56.025722 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb07ddff-beb4-443b-91bf-5757ff7a87f0-bound-sa-token\") pod \"cert-manager-597b95cb84-h9xf2\" (UID: \"cb07ddff-beb4-443b-91bf-5757ff7a87f0\") " pod="cert-manager/cert-manager-597b95cb84-h9xf2" Oct 30 18:42:56.025816 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:56.025769 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r26ml\" (UniqueName: \"kubernetes.io/projected/cb07ddff-beb4-443b-91bf-5757ff7a87f0-kube-api-access-r26ml\") pod \"cert-manager-597b95cb84-h9xf2\" (UID: \"cb07ddff-beb4-443b-91bf-5757ff7a87f0\") " pod="cert-manager/cert-manager-597b95cb84-h9xf2" Oct 30 18:42:56.126311 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:56.126274 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r26ml\" (UniqueName: \"kubernetes.io/projected/cb07ddff-beb4-443b-91bf-5757ff7a87f0-kube-api-access-r26ml\") pod \"cert-manager-597b95cb84-h9xf2\" (UID: \"cb07ddff-beb4-443b-91bf-5757ff7a87f0\") " pod="cert-manager/cert-manager-597b95cb84-h9xf2" Oct 30 18:42:56.126499 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:56.126338 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb07ddff-beb4-443b-91bf-5757ff7a87f0-bound-sa-token\") pod \"cert-manager-597b95cb84-h9xf2\" (UID: \"cb07ddff-beb4-443b-91bf-5757ff7a87f0\") " pod="cert-manager/cert-manager-597b95cb84-h9xf2" Oct 30 18:42:56.184099 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:56.184031 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cb07ddff-beb4-443b-91bf-5757ff7a87f0-bound-sa-token\") pod \"cert-manager-597b95cb84-h9xf2\" (UID: \"cb07ddff-beb4-443b-91bf-5757ff7a87f0\") " pod="cert-manager/cert-manager-597b95cb84-h9xf2" Oct 30 18:42:56.184293 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:56.184274 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r26ml\" (UniqueName: \"kubernetes.io/projected/cb07ddff-beb4-443b-91bf-5757ff7a87f0-kube-api-access-r26ml\") pod \"cert-manager-597b95cb84-h9xf2\" (UID: \"cb07ddff-beb4-443b-91bf-5757ff7a87f0\") " pod="cert-manager/cert-manager-597b95cb84-h9xf2" Oct 30 18:42:56.244834 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:56.244807 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-597b95cb84-h9xf2" Oct 30 18:42:56.609784 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:56.609709 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-597b95cb84-h9xf2"] Oct 30 18:42:56.613101 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:42:56.613072 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcb07ddff_beb4_443b_91bf_5757ff7a87f0.slice/crio-e5f3a57f4935248a0139989c42808185265d2e9cdb6e06d9fd5e5a354ad1967c WatchSource:0}: Error finding container e5f3a57f4935248a0139989c42808185265d2e9cdb6e06d9fd5e5a354ad1967c: Status 404 returned error can't find the container with id e5f3a57f4935248a0139989c42808185265d2e9cdb6e06d9fd5e5a354ad1967c Oct 30 18:42:56.621406 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:42:56.621384 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-h9xf2" event={"ID":"cb07ddff-beb4-443b-91bf-5757ff7a87f0","Type":"ContainerStarted","Data":"e5f3a57f4935248a0139989c42808185265d2e9cdb6e06d9fd5e5a354ad1967c"} Oct 30 18:43:01.646935 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:01.646899 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-597b95cb84-h9xf2" event={"ID":"cb07ddff-beb4-443b-91bf-5757ff7a87f0","Type":"ContainerStarted","Data":"3caac8a9c9d421648c95afc8e56919cde5888fa1a4ed3c55668dbf3ae764e1d8"} Oct 30 18:43:01.762258 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:01.762207 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-597b95cb84-h9xf2" podStartSLOduration=2.118402535 podStartE2EDuration="6.762192831s" totalImagesPullingTime="4.643790296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:42:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:01.761662993 +0000 UTC m=+252.251790816" watchObservedRunningTime="2026-10-30 18:43:01.762192831 +0000 UTC m=+252.252320648" Oct 30 18:43:02.554741 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:02.554704 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:02.727497 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:02.727458 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:04.734224 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:04.734187 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:07.758172 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:07.758134 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:13.051987 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:13.051951 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:15.834630 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:15.834566 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:16.006019 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.005978 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:16.595285 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.595254 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:16.765819 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.765788 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8"] Oct 30 18:43:16.769046 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.769030 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:16.853166 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.853080 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Oct 30 18:43:16.853510 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.853166 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Oct 30 18:43:16.855612 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.855591 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Oct 30 18:43:16.855759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.855617 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Oct 30 18:43:16.875644 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.875619 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-metrics-tls\"" Oct 30 18:43:16.875644 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.875619 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-wzclj\"" Oct 30 18:43:16.882127 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.882103 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2b396120-b7c7-4164-b320-4e4427016800-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cb69c88fb-9fph8\" (UID: \"2b396120-b7c7-4164-b320-4e4427016800\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:16.882226 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.882151 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b396120-b7c7-4164-b320-4e4427016800-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cb69c88fb-9fph8\" (UID: \"2b396120-b7c7-4164-b320-4e4427016800\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:16.882226 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.882199 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b396120-b7c7-4164-b320-4e4427016800-metrics-cert\") pod \"opendatahub-operator-controller-manager-7cb69c88fb-9fph8\" (UID: \"2b396120-b7c7-4164-b320-4e4427016800\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:16.882307 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.882252 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8r97\" (UniqueName: \"kubernetes.io/projected/2b396120-b7c7-4164-b320-4e4427016800-kube-api-access-q8r97\") pod \"opendatahub-operator-controller-manager-7cb69c88fb-9fph8\" (UID: \"2b396120-b7c7-4164-b320-4e4427016800\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:16.882307 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.882271 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b396120-b7c7-4164-b320-4e4427016800-tmp\") pod \"opendatahub-operator-controller-manager-7cb69c88fb-9fph8\" (UID: \"2b396120-b7c7-4164-b320-4e4427016800\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:16.882307 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.882296 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2b396120-b7c7-4164-b320-4e4427016800-manifests\") pod \"opendatahub-operator-controller-manager-7cb69c88fb-9fph8\" (UID: \"2b396120-b7c7-4164-b320-4e4427016800\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:16.912785 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.912766 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:16.937566 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.937535 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8"] Oct 30 18:43:16.983744 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.983718 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2b396120-b7c7-4164-b320-4e4427016800-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cb69c88fb-9fph8\" (UID: \"2b396120-b7c7-4164-b320-4e4427016800\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:16.983871 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.983756 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b396120-b7c7-4164-b320-4e4427016800-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cb69c88fb-9fph8\" (UID: \"2b396120-b7c7-4164-b320-4e4427016800\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:16.983871 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.983777 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b396120-b7c7-4164-b320-4e4427016800-metrics-cert\") pod \"opendatahub-operator-controller-manager-7cb69c88fb-9fph8\" (UID: \"2b396120-b7c7-4164-b320-4e4427016800\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:16.983871 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.983811 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q8r97\" (UniqueName: \"kubernetes.io/projected/2b396120-b7c7-4164-b320-4e4427016800-kube-api-access-q8r97\") pod \"opendatahub-operator-controller-manager-7cb69c88fb-9fph8\" (UID: \"2b396120-b7c7-4164-b320-4e4427016800\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:16.984031 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.983917 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b396120-b7c7-4164-b320-4e4427016800-tmp\") pod \"opendatahub-operator-controller-manager-7cb69c88fb-9fph8\" (UID: \"2b396120-b7c7-4164-b320-4e4427016800\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:16.984031 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.983964 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2b396120-b7c7-4164-b320-4e4427016800-manifests\") pod \"opendatahub-operator-controller-manager-7cb69c88fb-9fph8\" (UID: \"2b396120-b7c7-4164-b320-4e4427016800\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:16.984358 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.984331 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/2b396120-b7c7-4164-b320-4e4427016800-manifests\") pod \"opendatahub-operator-controller-manager-7cb69c88fb-9fph8\" (UID: \"2b396120-b7c7-4164-b320-4e4427016800\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:16.986185 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.986163 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/2b396120-b7c7-4164-b320-4e4427016800-tmp\") pod \"opendatahub-operator-controller-manager-7cb69c88fb-9fph8\" (UID: \"2b396120-b7c7-4164-b320-4e4427016800\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:16.986296 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.986280 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2b396120-b7c7-4164-b320-4e4427016800-metrics-cert\") pod \"opendatahub-operator-controller-manager-7cb69c88fb-9fph8\" (UID: \"2b396120-b7c7-4164-b320-4e4427016800\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:16.986409 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.986385 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2b396120-b7c7-4164-b320-4e4427016800-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cb69c88fb-9fph8\" (UID: \"2b396120-b7c7-4164-b320-4e4427016800\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:16.986538 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:16.986518 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/2b396120-b7c7-4164-b320-4e4427016800-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cb69c88fb-9fph8\" (UID: \"2b396120-b7c7-4164-b320-4e4427016800\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:17.044122 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:17.044088 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8r97\" (UniqueName: \"kubernetes.io/projected/2b396120-b7c7-4164-b320-4e4427016800-kube-api-access-q8r97\") pod \"opendatahub-operator-controller-manager-7cb69c88fb-9fph8\" (UID: \"2b396120-b7c7-4164-b320-4e4427016800\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:17.082936 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:17.082905 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:17.117681 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:17.117561 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:17.310265 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:17.310234 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:17.649542 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:17.649520 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8"] Oct 30 18:43:17.650860 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:43:17.650835 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b396120_b7c7_4164_b320_4e4427016800.slice/crio-cfa8fb04a025ffbaf26cb5b713d138e52e99717419ecdd7bf2eecaa7fd98b536 WatchSource:0}: Error finding container cfa8fb04a025ffbaf26cb5b713d138e52e99717419ecdd7bf2eecaa7fd98b536: Status 404 returned error can't find the container with id cfa8fb04a025ffbaf26cb5b713d138e52e99717419ecdd7bf2eecaa7fd98b536 Oct 30 18:43:17.700095 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:17.700062 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" event={"ID":"2b396120-b7c7-4164-b320-4e4427016800","Type":"ContainerStarted","Data":"cfa8fb04a025ffbaf26cb5b713d138e52e99717419ecdd7bf2eecaa7fd98b536"} Oct 30 18:43:20.120437 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:43:20.120396 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:43:21.719238 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:21.719155 5894 generic.go:356] "Generic (PLEG): container finished" podID="2b396120-b7c7-4164-b320-4e4427016800" containerID="27c9be679b318cc59c4a58383219ac1207e8f3fdf3ed70e6154e90312d9080c0" exitCode=0 Oct 30 18:43:21.719656 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:21.719240 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" event={"ID":"2b396120-b7c7-4164-b320-4e4427016800","Type":"ContainerDied","Data":"27c9be679b318cc59c4a58383219ac1207e8f3fdf3ed70e6154e90312d9080c0"} Oct 30 18:43:22.726222 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:22.726189 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" event={"ID":"2b396120-b7c7-4164-b320-4e4427016800","Type":"ContainerStarted","Data":"829da1467a48c473b79f3942611643e21658c732b4e4d32a1af5602bcf047b59"} Oct 30 18:43:22.726613 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:22.726356 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:22.952919 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:22.952861 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" podStartSLOduration=2.43491763 podStartE2EDuration="6.952843992s" totalImagesPullingTime="4.04903116s" totalInitContainerRuntime="468.895202ms" isStatefulPod=false podCreationTimestamp="2026-10-30 18:43:16 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:22.903165205 +0000 UTC m=+273.393293064" watchObservedRunningTime="2026-10-30 18:43:22.952843992 +0000 UTC m=+273.442971805" Oct 30 18:43:23.845358 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:23.845319 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:24.017010 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:24.016978 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:24.196774 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:24.196729 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:27.005975 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:27.005940 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:33.732008 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:33.731937 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7cb69c88fb-9fph8" Oct 30 18:43:38.903209 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:38.903174 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:39.123956 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:39.123922 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:40.194963 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:40.194931 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-5fb96896c4-jfftn"] Oct 30 18:43:40.198233 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:40.198213 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5fb96896c4-jfftn" Oct 30 18:43:40.199017 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:40.198987 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:40.267914 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:40.267886 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914-tls-certs\") pod \"kube-auth-proxy-5fb96896c4-jfftn\" (UID: \"e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914\") " pod="openshift-ingress/kube-auth-proxy-5fb96896c4-jfftn" Oct 30 18:43:40.268042 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:40.267942 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vhmpr\" (UniqueName: \"kubernetes.io/projected/e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914-kube-api-access-vhmpr\") pod \"kube-auth-proxy-5fb96896c4-jfftn\" (UID: \"e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914\") " pod="openshift-ingress/kube-auth-proxy-5fb96896c4-jfftn" Oct 30 18:43:40.268085 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:40.268034 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914-tmp\") pod \"kube-auth-proxy-5fb96896c4-jfftn\" (UID: \"e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914\") " pod="openshift-ingress/kube-auth-proxy-5fb96896c4-jfftn" Oct 30 18:43:40.301194 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:40.301158 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5fb96896c4-jfftn"] Oct 30 18:43:40.306556 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:40.306528 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Oct 30 18:43:40.306688 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:40.306638 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-4w8lc\"" Oct 30 18:43:40.334367 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:40.334343 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Oct 30 18:43:40.368797 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:40.368771 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914-tmp\") pod \"kube-auth-proxy-5fb96896c4-jfftn\" (UID: \"e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914\") " pod="openshift-ingress/kube-auth-proxy-5fb96896c4-jfftn" Oct 30 18:43:40.368902 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:40.368807 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914-tls-certs\") pod \"kube-auth-proxy-5fb96896c4-jfftn\" (UID: \"e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914\") " pod="openshift-ingress/kube-auth-proxy-5fb96896c4-jfftn" Oct 30 18:43:40.368902 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:40.368848 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vhmpr\" (UniqueName: \"kubernetes.io/projected/e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914-kube-api-access-vhmpr\") pod \"kube-auth-proxy-5fb96896c4-jfftn\" (UID: \"e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914\") " pod="openshift-ingress/kube-auth-proxy-5fb96896c4-jfftn" Oct 30 18:43:40.371144 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:40.371118 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914-tmp\") pod \"kube-auth-proxy-5fb96896c4-jfftn\" (UID: \"e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914\") " pod="openshift-ingress/kube-auth-proxy-5fb96896c4-jfftn" Oct 30 18:43:40.371259 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:40.371170 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914-tls-certs\") pod \"kube-auth-proxy-5fb96896c4-jfftn\" (UID: \"e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914\") " pod="openshift-ingress/kube-auth-proxy-5fb96896c4-jfftn" Oct 30 18:43:40.427410 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:40.427382 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vhmpr\" (UniqueName: \"kubernetes.io/projected/e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914-kube-api-access-vhmpr\") pod \"kube-auth-proxy-5fb96896c4-jfftn\" (UID: \"e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914\") " pod="openshift-ingress/kube-auth-proxy-5fb96896c4-jfftn" Oct 30 18:43:40.471360 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:40.471297 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:40.510042 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:40.510014 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-5fb96896c4-jfftn" Oct 30 18:43:40.882514 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:40.882433 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-5fb96896c4-jfftn"] Oct 30 18:43:40.886072 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:43:40.886028 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1d14c98_aefa_45c8_a6c2_8dcdfc2e3914.slice/crio-a5e473ff998701f6ed0485b8222f73c9c2c5cd4b2c0b4fa3daa1291257d0ee7d WatchSource:0}: Error finding container a5e473ff998701f6ed0485b8222f73c9c2c5cd4b2c0b4fa3daa1291257d0ee7d: Status 404 returned error can't find the container with id a5e473ff998701f6ed0485b8222f73c9c2c5cd4b2c0b4fa3daa1291257d0ee7d Oct 30 18:43:41.491782 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.491750 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc"] Oct 30 18:43:41.496128 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.496106 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.550856 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.550820 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Oct 30 18:43:41.550856 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.550830 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Oct 30 18:43:41.551043 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.551008 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Oct 30 18:43:41.551043 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.551015 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Oct 30 18:43:41.551131 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.551055 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-5sqr9\"" Oct 30 18:43:41.567392 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.567372 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:41.578700 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.578677 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.578808 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.578717 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.578808 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.578745 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.578877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.578807 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.578877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.578842 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4nd5\" (UniqueName: \"kubernetes.io/projected/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-kube-api-access-r4nd5\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.578997 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.578951 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.579033 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.579016 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.601278 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.601255 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc"] Oct 30 18:43:41.679463 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.679435 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.679623 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.679469 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.679623 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.679485 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4nd5\" (UniqueName: \"kubernetes.io/projected/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-kube-api-access-r4nd5\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.679623 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.679511 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.679623 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.679550 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.679623 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.679615 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.679885 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.679662 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.680240 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.680210 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.681776 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.681750 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-local-certs\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.682371 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.682349 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-cacerts\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.682446 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.682404 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.682507 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.682488 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-kubeconfig\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.736136 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.736104 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:41.736308 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.736290 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-token\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.736540 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.736521 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4nd5\" (UniqueName: \"kubernetes.io/projected/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-kube-api-access-r4nd5\") pod \"istiod-openshift-gateway-7876bfcb74-gjbpc\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:41.802536 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.802473 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5fb96896c4-jfftn" event={"ID":"e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914","Type":"ContainerStarted","Data":"a5e473ff998701f6ed0485b8222f73c9c2c5cd4b2c0b4fa3daa1291257d0ee7d"} Oct 30 18:43:41.807700 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:41.807671 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:42.228334 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:42.228255 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc"] Oct 30 18:43:42.230861 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:43:42.230833 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18e818b3_4e93_4fbd_b7e8_f3c592d98b97.slice/crio-c0ea90e80eecd457084bb5190ceca844d53c18fed42f92ea35553c9e92b0821f WatchSource:0}: Error finding container c0ea90e80eecd457084bb5190ceca844d53c18fed42f92ea35553c9e92b0821f: Status 404 returned error can't find the container with id c0ea90e80eecd457084bb5190ceca844d53c18fed42f92ea35553c9e92b0821f Oct 30 18:43:42.565674 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:42.565598 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:42.808172 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:42.808138 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" event={"ID":"18e818b3-4e93-4fbd-b7e8-f3c592d98b97","Type":"ContainerStarted","Data":"c0ea90e80eecd457084bb5190ceca844d53c18fed42f92ea35553c9e92b0821f"} Oct 30 18:43:43.813895 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:43.813859 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-5fb96896c4-jfftn" event={"ID":"e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914","Type":"ContainerStarted","Data":"b73125a5aa1602e90fa61caacdf2fa6d2cfe0260d10a39a586c54c2a40aa5d60"} Oct 30 18:43:43.932648 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:43.932599 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-5fb96896c4-jfftn" podStartSLOduration=1.185048777 podStartE2EDuration="3.932563507s" totalImagesPullingTime="2.74751473s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:43:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:43.932562356 +0000 UTC m=+294.422690188" watchObservedRunningTime="2026-10-30 18:43:43.932563507 +0000 UTC m=+294.422691324" Oct 30 18:43:46.372125 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:46.372094 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:43:46.372384 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:46.372156 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:43:46.828075 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:46.828025 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" event={"ID":"18e818b3-4e93-4fbd-b7e8-f3c592d98b97","Type":"ContainerStarted","Data":"1a7aec344f677ac360502d2e117341f3657ab45c16f3c0a423170c38e327fd36"} Oct 30 18:43:46.828259 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:46.828178 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:47.044183 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:47.044135 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" podStartSLOduration=1.905973286 podStartE2EDuration="6.044121131s" totalImagesPullingTime="4.138147845s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:43:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:43:46.994017907 +0000 UTC m=+297.484145722" watchObservedRunningTime="2026-10-30 18:43:47.044121131 +0000 UTC m=+297.534248948" Oct 30 18:43:47.555974 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:47.555936 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:47.733383 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:47.733338 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:47.834181 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:47.834100 5894 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7876bfcb74-gjbpc container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 18:43:47.834344 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:47.834167 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" podUID="18e818b3-4e93-4fbd-b7e8-f3c592d98b97" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 18:43:48.487152 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:48.487117 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:48.695956 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:48.695926 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:49.432026 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:49.431996 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:49.675465 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:49.675433 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 18:43:49.679939 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:49.679913 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 18:43:49.740642 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:49.740547 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:50.116965 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:43:50.116882 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:43:50.625910 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:50.625854 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:43:50.832924 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:50.832898 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:43:51.395908 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:43:51.395870 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:06.979913 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:06.979881 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:09.154894 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:09.154860 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:13.391036 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:13.390994 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:19.169353 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:19.169320 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:19.339223 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:19.339188 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:44:20.116727 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:44:20.116686 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:44:50.132142 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:44:50.132101 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:44:58.575098 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.575056 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8"] Oct 30 18:44:58.577488 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.577464 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.683030 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.682994 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8"] Oct 30 18:44:58.713087 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.713059 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp79c\" (UniqueName: \"kubernetes.io/projected/55214cf0-106f-45de-953c-fca931bc6a4f-kube-api-access-cp79c\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.713270 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.713096 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/55214cf0-106f-45de-953c-fca931bc6a4f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.713270 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.713125 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/55214cf0-106f-45de-953c-fca931bc6a4f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.713270 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.713245 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55214cf0-106f-45de-953c-fca931bc6a4f-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.713402 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.713319 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/55214cf0-106f-45de-953c-fca931bc6a4f-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.713402 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.713349 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/55214cf0-106f-45de-953c-fca931bc6a4f-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.713402 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.713388 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/55214cf0-106f-45de-953c-fca931bc6a4f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.814066 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.814027 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/55214cf0-106f-45de-953c-fca931bc6a4f-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.814066 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.814063 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/55214cf0-106f-45de-953c-fca931bc6a4f-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.814317 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.814085 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/55214cf0-106f-45de-953c-fca931bc6a4f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.814317 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.814144 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cp79c\" (UniqueName: \"kubernetes.io/projected/55214cf0-106f-45de-953c-fca931bc6a4f-kube-api-access-cp79c\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.814317 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.814185 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/55214cf0-106f-45de-953c-fca931bc6a4f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.814317 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.814229 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/55214cf0-106f-45de-953c-fca931bc6a4f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.814317 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.814281 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55214cf0-106f-45de-953c-fca931bc6a4f-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.815285 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.815257 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/55214cf0-106f-45de-953c-fca931bc6a4f-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.816757 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.816736 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/55214cf0-106f-45de-953c-fca931bc6a4f-cacerts\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.816950 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.816930 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/55214cf0-106f-45de-953c-fca931bc6a4f-istio-kubeconfig\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.817044 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.817022 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/55214cf0-106f-45de-953c-fca931bc6a4f-local-certs\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.817085 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.817044 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/55214cf0-106f-45de-953c-fca931bc6a4f-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.873435 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.873371 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/55214cf0-106f-45de-953c-fca931bc6a4f-istio-token\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.873542 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.873449 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp79c\" (UniqueName: \"kubernetes.io/projected/55214cf0-106f-45de-953c-fca931bc6a4f-kube-api-access-cp79c\") pod \"istiod-openshift-gateway-7ddd9cfdc8-n65j8\" (UID: \"55214cf0-106f-45de-953c-fca931bc6a4f\") " pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:58.887771 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:58.887749 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:44:59.312534 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:59.312436 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8"] Oct 30 18:44:59.315314 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:44:59.315287 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55214cf0_106f_45de_953c_fca931bc6a4f.slice/crio-f63fcfa2a79eccea0de2e39f89a4855ca94aa8fa7d5efa0fa41d81c36c92e230 WatchSource:0}: Error finding container f63fcfa2a79eccea0de2e39f89a4855ca94aa8fa7d5efa0fa41d81c36c92e230: Status 404 returned error can't find the container with id f63fcfa2a79eccea0de2e39f89a4855ca94aa8fa7d5efa0fa41d81c36c92e230 Oct 30 18:44:59.317351 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:59.317323 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:44:59.317426 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:44:59.317383 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:45:00.120008 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:00.119973 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" event={"ID":"55214cf0-106f-45de-953c-fca931bc6a4f","Type":"ContainerStarted","Data":"da3791b03e65a543523ddcf97a1aca2dfc6addbd9d7d7c21c83a2ed21c1f8b2d"} Oct 30 18:45:00.120445 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:00.120021 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" event={"ID":"55214cf0-106f-45de-953c-fca931bc6a4f","Type":"ContainerStarted","Data":"f63fcfa2a79eccea0de2e39f89a4855ca94aa8fa7d5efa0fa41d81c36c92e230"} Oct 30 18:45:00.120445 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:00.120210 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:45:00.121433 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:00.121406 5894 patch_prober.go:29] interesting pod/istiod-openshift-gateway-7ddd9cfdc8-n65j8 container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 18:45:00.121558 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:00.121451 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" podUID="55214cf0-106f-45de-953c-fca931bc6a4f" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 18:45:00.239914 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:00.239862 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" podStartSLOduration=2.239848269 podStartE2EDuration="2.239848269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:44:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:00.239666483 +0000 UTC m=+370.729794304" watchObservedRunningTime="2026-10-30 18:45:00.239848269 +0000 UTC m=+370.729976089" Oct 30 18:45:01.125015 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.124941 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7ddd9cfdc8-n65j8" Oct 30 18:45:01.343363 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.343319 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc"] Oct 30 18:45:01.343587 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.343550 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" podUID="18e818b3-4e93-4fbd-b7e8-f3c592d98b97" containerName="discovery" containerID="cri-o://1a7aec344f677ac360502d2e117341f3657ab45c16f3c0a423170c38e327fd36" gracePeriod=30 Oct 30 18:45:01.584468 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.584440 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:45:01.740057 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.739972 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-kubeconfig\") pod \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " Oct 30 18:45:01.740057 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.740010 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-csr-ca-configmap\") pod \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " Oct 30 18:45:01.740057 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.740033 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-local-certs\" (UniqueName: \"kubernetes.io/empty-dir/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-local-certs\") pod \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " Oct 30 18:45:01.740327 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.740063 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-csr-dns-cert\") pod \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " Oct 30 18:45:01.740327 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.740091 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-kube-api-access-r4nd5\" (UniqueName: \"kubernetes.io/projected/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-kube-api-access-r4nd5\") pod \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " Oct 30 18:45:01.740327 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.740115 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-token\" (UniqueName: \"kubernetes.io/projected/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-token\") pod \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " Oct 30 18:45:01.740327 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.740141 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-cacerts\" (UniqueName: \"kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-cacerts\") pod \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\" (UID: \"18e818b3-4e93-4fbd-b7e8-f3c592d98b97\") " Oct 30 18:45:01.740625 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.740488 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-csr-ca-configmap" pod "18e818b3-4e93-4fbd-b7e8-f3c592d98b97" (UID: "18e818b3-4e93-4fbd-b7e8-f3c592d98b97"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:45:01.742653 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.742606 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-csr-dns-cert" pod "18e818b3-4e93-4fbd-b7e8-f3c592d98b97" (UID: "18e818b3-4e93-4fbd-b7e8-f3c592d98b97"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:45:01.742792 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.742744 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-kube-api-access-r4nd5" pod "18e818b3-4e93-4fbd-b7e8-f3c592d98b97" (UID: "18e818b3-4e93-4fbd-b7e8-f3c592d98b97"). InnerVolumeSpecName "kube-api-access-r4nd5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:45:01.742860 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.742808 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-token" pod "18e818b3-4e93-4fbd-b7e8-f3c592d98b97" (UID: "18e818b3-4e93-4fbd-b7e8-f3c592d98b97"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:45:01.742915 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.742897 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-kubeconfig" pod "18e818b3-4e93-4fbd-b7e8-f3c592d98b97" (UID: "18e818b3-4e93-4fbd-b7e8-f3c592d98b97"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:45:01.742970 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.742907 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-local-certs" pod "18e818b3-4e93-4fbd-b7e8-f3c592d98b97" (UID: "18e818b3-4e93-4fbd-b7e8-f3c592d98b97"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:45:01.742970 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.742914 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-cacerts" pod "18e818b3-4e93-4fbd-b7e8-f3c592d98b97" (UID: "18e818b3-4e93-4fbd-b7e8-f3c592d98b97"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:45:01.842303 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.841011 5894 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-kubeconfig\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:45:01.842303 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.841051 5894 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-csr-ca-configmap\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:45:01.842303 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.841067 5894 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-local-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:45:01.842303 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.841083 5894 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-csr-dns-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:45:01.842303 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.841096 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r4nd5\" (UniqueName: \"kubernetes.io/projected/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-kube-api-access-r4nd5\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:45:01.842303 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.841110 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:45:01.842303 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:01.841123 5894 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/18e818b3-4e93-4fbd-b7e8-f3c592d98b97-cacerts\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:45:02.129227 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:02.129141 5894 generic.go:356] "Generic (PLEG): container finished" podID="18e818b3-4e93-4fbd-b7e8-f3c592d98b97" containerID="1a7aec344f677ac360502d2e117341f3657ab45c16f3c0a423170c38e327fd36" exitCode=0 Oct 30 18:45:02.129227 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:02.129190 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" Oct 30 18:45:02.129676 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:02.129219 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" event={"ID":"18e818b3-4e93-4fbd-b7e8-f3c592d98b97","Type":"ContainerDied","Data":"1a7aec344f677ac360502d2e117341f3657ab45c16f3c0a423170c38e327fd36"} Oct 30 18:45:02.129676 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:02.129268 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc" event={"ID":"18e818b3-4e93-4fbd-b7e8-f3c592d98b97","Type":"ContainerDied","Data":"c0ea90e80eecd457084bb5190ceca844d53c18fed42f92ea35553c9e92b0821f"} Oct 30 18:45:02.129676 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:02.129291 5894 scope.go:122] "RemoveContainer" containerID="1a7aec344f677ac360502d2e117341f3657ab45c16f3c0a423170c38e327fd36" Oct 30 18:45:02.137932 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:02.137912 5894 scope.go:122] "RemoveContainer" containerID="1a7aec344f677ac360502d2e117341f3657ab45c16f3c0a423170c38e327fd36" Oct 30 18:45:02.138197 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:45:02.138177 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a7aec344f677ac360502d2e117341f3657ab45c16f3c0a423170c38e327fd36\": container with ID starting with 1a7aec344f677ac360502d2e117341f3657ab45c16f3c0a423170c38e327fd36 not found: ID does not exist" containerID="1a7aec344f677ac360502d2e117341f3657ab45c16f3c0a423170c38e327fd36" Oct 30 18:45:02.138261 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:02.138210 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a7aec344f677ac360502d2e117341f3657ab45c16f3c0a423170c38e327fd36"} err="failed to get container status \"1a7aec344f677ac360502d2e117341f3657ab45c16f3c0a423170c38e327fd36\": rpc error: code = NotFound desc = could not find container \"1a7aec344f677ac360502d2e117341f3657ab45c16f3c0a423170c38e327fd36\": container with ID starting with 1a7aec344f677ac360502d2e117341f3657ab45c16f3c0a423170c38e327fd36 not found: ID does not exist" Oct 30 18:45:02.290953 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:02.290916 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc"] Oct 30 18:45:02.340551 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:02.340513 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7876bfcb74-gjbpc"] Oct 30 18:45:03.648294 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:03.648259 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="18e818b3-4e93-4fbd-b7e8-f3c592d98b97" path="/var/lib/kubelet/pods/18e818b3-4e93-4fbd-b7e8-f3c592d98b97/volumes" Oct 30 18:45:03.768321 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:03.768289 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:03.943149 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:03.943070 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:04.115308 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.115272 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:04.285641 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.285536 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:04.457494 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.457463 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:04.585817 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.585737 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-zf2dd"] Oct 30 18:45:04.586128 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.586113 5894 cpu_manager.go:414] "containerMap: removing container" podUID="18e818b3-4e93-4fbd-b7e8-f3c592d98b97" containerName="discovery" Oct 30 18:45:04.586169 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.586131 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="18e818b3-4e93-4fbd-b7e8-f3c592d98b97" containerName="discovery" Oct 30 18:45:04.586200 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.586195 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="18e818b3-4e93-4fbd-b7e8-f3c592d98b97" containerName="discovery" Oct 30 18:45:04.590425 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.590402 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-zf2dd" Oct 30 18:45:04.628280 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.628244 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:04.644951 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.644916 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-zf2dd"] Oct 30 18:45:04.693004 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.692974 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-qvp68\"" Oct 30 18:45:04.693004 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.692996 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Oct 30 18:45:04.695873 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.695835 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256"] Oct 30 18:45:04.699042 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.699026 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256" Oct 30 18:45:04.751267 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.751234 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-gfqbv\"" Oct 30 18:45:04.751425 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.751292 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Oct 30 18:45:04.756089 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.756064 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256"] Oct 30 18:45:04.765222 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.765192 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-86d72\" (UniqueName: \"kubernetes.io/projected/ef60d381-c326-449b-a5f5-47b539535199-kube-api-access-86d72\") pod \"kserve-controller-manager-776cc66d47-zf2dd\" (UID: \"ef60d381-c326-449b-a5f5-47b539535199\") " pod="opendatahub/kserve-controller-manager-776cc66d47-zf2dd" Oct 30 18:45:04.765353 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.765239 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef60d381-c326-449b-a5f5-47b539535199-cert\") pod \"kserve-controller-manager-776cc66d47-zf2dd\" (UID: \"ef60d381-c326-449b-a5f5-47b539535199\") " pod="opendatahub/kserve-controller-manager-776cc66d47-zf2dd" Oct 30 18:45:04.807271 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.807240 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-b2qh7"] Oct 30 18:45:04.810495 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.810471 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:04.810644 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.810628 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-b2qh7" Oct 30 18:45:04.863023 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.862959 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-cw4pd\"" Oct 30 18:45:04.863154 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.863034 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Oct 30 18:45:04.866138 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.866114 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-86d72\" (UniqueName: \"kubernetes.io/projected/ef60d381-c326-449b-a5f5-47b539535199-kube-api-access-86d72\") pod \"kserve-controller-manager-776cc66d47-zf2dd\" (UID: \"ef60d381-c326-449b-a5f5-47b539535199\") " pod="opendatahub/kserve-controller-manager-776cc66d47-zf2dd" Oct 30 18:45:04.866254 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.866170 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ee74582d-1edf-412c-9a76-459d08a10cc3-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-ql256\" (UID: \"ee74582d-1edf-412c-9a76-459d08a10cc3\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256" Oct 30 18:45:04.866330 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.866305 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef60d381-c326-449b-a5f5-47b539535199-cert\") pod \"kserve-controller-manager-776cc66d47-zf2dd\" (UID: \"ef60d381-c326-449b-a5f5-47b539535199\") " pod="opendatahub/kserve-controller-manager-776cc66d47-zf2dd" Oct 30 18:45:04.866510 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.866487 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt5zq\" (UniqueName: \"kubernetes.io/projected/ee74582d-1edf-412c-9a76-459d08a10cc3-kube-api-access-jt5zq\") pod \"llmisvc-controller-manager-869d5dcc5d-ql256\" (UID: \"ee74582d-1edf-412c-9a76-459d08a10cc3\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256" Oct 30 18:45:04.867449 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.867350 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-b2qh7"] Oct 30 18:45:04.869194 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.869176 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ef60d381-c326-449b-a5f5-47b539535199-cert\") pod \"kserve-controller-manager-776cc66d47-zf2dd\" (UID: \"ef60d381-c326-449b-a5f5-47b539535199\") " pod="opendatahub/kserve-controller-manager-776cc66d47-zf2dd" Oct 30 18:45:04.925513 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.925479 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-86d72\" (UniqueName: \"kubernetes.io/projected/ef60d381-c326-449b-a5f5-47b539535199-kube-api-access-86d72\") pod \"kserve-controller-manager-776cc66d47-zf2dd\" (UID: \"ef60d381-c326-449b-a5f5-47b539535199\") " pod="opendatahub/kserve-controller-manager-776cc66d47-zf2dd" Oct 30 18:45:04.967953 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.967919 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ee74582d-1edf-412c-9a76-459d08a10cc3-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-ql256\" (UID: \"ee74582d-1edf-412c-9a76-459d08a10cc3\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256" Oct 30 18:45:04.967953 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.967957 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/310c3e75-e27c-4094-b0f6-97f99741a1d6-tls-certs\") pod \"model-serving-api-75bd89fdfc-b2qh7\" (UID: \"310c3e75-e27c-4094-b0f6-97f99741a1d6\") " pod="opendatahub/model-serving-api-75bd89fdfc-b2qh7" Oct 30 18:45:04.968155 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.968080 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt5zq\" (UniqueName: \"kubernetes.io/projected/ee74582d-1edf-412c-9a76-459d08a10cc3-kube-api-access-jt5zq\") pod \"llmisvc-controller-manager-869d5dcc5d-ql256\" (UID: \"ee74582d-1edf-412c-9a76-459d08a10cc3\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256" Oct 30 18:45:04.968220 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.968198 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlp45\" (UniqueName: \"kubernetes.io/projected/310c3e75-e27c-4094-b0f6-97f99741a1d6-kube-api-access-rlp45\") pod \"model-serving-api-75bd89fdfc-b2qh7\" (UID: \"310c3e75-e27c-4094-b0f6-97f99741a1d6\") " pod="opendatahub/model-serving-api-75bd89fdfc-b2qh7" Oct 30 18:45:04.970278 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:04.970260 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ee74582d-1edf-412c-9a76-459d08a10cc3-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-ql256\" (UID: \"ee74582d-1edf-412c-9a76-459d08a10cc3\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256" Oct 30 18:45:05.025511 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:05.025482 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt5zq\" (UniqueName: \"kubernetes.io/projected/ee74582d-1edf-412c-9a76-459d08a10cc3-kube-api-access-jt5zq\") pod \"llmisvc-controller-manager-869d5dcc5d-ql256\" (UID: \"ee74582d-1edf-412c-9a76-459d08a10cc3\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256" Oct 30 18:45:05.069562 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:05.069525 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/310c3e75-e27c-4094-b0f6-97f99741a1d6-tls-certs\") pod \"model-serving-api-75bd89fdfc-b2qh7\" (UID: \"310c3e75-e27c-4094-b0f6-97f99741a1d6\") " pod="opendatahub/model-serving-api-75bd89fdfc-b2qh7" Oct 30 18:45:05.069751 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:05.069636 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlp45\" (UniqueName: \"kubernetes.io/projected/310c3e75-e27c-4094-b0f6-97f99741a1d6-kube-api-access-rlp45\") pod \"model-serving-api-75bd89fdfc-b2qh7\" (UID: \"310c3e75-e27c-4094-b0f6-97f99741a1d6\") " pod="opendatahub/model-serving-api-75bd89fdfc-b2qh7" Oct 30 18:45:05.071974 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:05.071950 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/310c3e75-e27c-4094-b0f6-97f99741a1d6-tls-certs\") pod \"model-serving-api-75bd89fdfc-b2qh7\" (UID: \"310c3e75-e27c-4094-b0f6-97f99741a1d6\") " pod="opendatahub/model-serving-api-75bd89fdfc-b2qh7" Oct 30 18:45:05.127759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:05.127684 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlp45\" (UniqueName: \"kubernetes.io/projected/310c3e75-e27c-4094-b0f6-97f99741a1d6-kube-api-access-rlp45\") pod \"model-serving-api-75bd89fdfc-b2qh7\" (UID: \"310c3e75-e27c-4094-b0f6-97f99741a1d6\") " pod="opendatahub/model-serving-api-75bd89fdfc-b2qh7" Oct 30 18:45:05.203133 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:05.203091 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-776cc66d47-zf2dd" Oct 30 18:45:05.310923 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:05.310890 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256" Oct 30 18:45:05.423232 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:05.423196 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-75bd89fdfc-b2qh7" Oct 30 18:45:05.569016 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:05.568991 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-776cc66d47-zf2dd"] Oct 30 18:45:05.571511 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:45:05.571483 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef60d381_c326_449b_a5f5_47b539535199.slice/crio-528c110fb0ed28eb751dca22513d74550cff67581669a58bd73f2415d173116c WatchSource:0}: Error finding container 528c110fb0ed28eb751dca22513d74550cff67581669a58bd73f2415d173116c: Status 404 returned error can't find the container with id 528c110fb0ed28eb751dca22513d74550cff67581669a58bd73f2415d173116c Oct 30 18:45:05.572822 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:05.572803 5894 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:45:05.674843 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:05.674754 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256"] Oct 30 18:45:05.677993 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:45:05.677961 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podee74582d_1edf_412c_9a76_459d08a10cc3.slice/crio-4c96a83f00ec403b066e510f05196323b7ea85e89fa85632eb5efc73962d7244 WatchSource:0}: Error finding container 4c96a83f00ec403b066e510f05196323b7ea85e89fa85632eb5efc73962d7244: Status 404 returned error can't find the container with id 4c96a83f00ec403b066e510f05196323b7ea85e89fa85632eb5efc73962d7244 Oct 30 18:45:05.790051 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:05.790022 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-75bd89fdfc-b2qh7"] Oct 30 18:45:05.793182 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:45:05.793156 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod310c3e75_e27c_4094_b0f6_97f99741a1d6.slice/crio-d45a54279ee3f490b5c6f20c08abc7e7fa97662f8f8c0f7e3d27da5b9b3beed4 WatchSource:0}: Error finding container d45a54279ee3f490b5c6f20c08abc7e7fa97662f8f8c0f7e3d27da5b9b3beed4: Status 404 returned error can't find the container with id d45a54279ee3f490b5c6f20c08abc7e7fa97662f8f8c0f7e3d27da5b9b3beed4 Oct 30 18:45:05.964715 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:05.964637 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:06.147832 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:06.147788 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-b2qh7" event={"ID":"310c3e75-e27c-4094-b0f6-97f99741a1d6","Type":"ContainerStarted","Data":"d45a54279ee3f490b5c6f20c08abc7e7fa97662f8f8c0f7e3d27da5b9b3beed4"} Oct 30 18:45:06.148907 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:06.148881 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-zf2dd" event={"ID":"ef60d381-c326-449b-a5f5-47b539535199","Type":"ContainerStarted","Data":"528c110fb0ed28eb751dca22513d74550cff67581669a58bd73f2415d173116c"} Oct 30 18:45:06.149967 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:06.149939 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256" event={"ID":"ee74582d-1edf-412c-9a76-459d08a10cc3","Type":"ContainerStarted","Data":"4c96a83f00ec403b066e510f05196323b7ea85e89fa85632eb5efc73962d7244"} Oct 30 18:45:09.168220 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:09.168191 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-75bd89fdfc-b2qh7" event={"ID":"310c3e75-e27c-4094-b0f6-97f99741a1d6","Type":"ContainerStarted","Data":"4619841e6eac9db8ca352fdc16a774f10c79d1fdb555a2b91e47a21fe6b224d1"} Oct 30 18:45:09.168607 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:09.168310 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-75bd89fdfc-b2qh7" Oct 30 18:45:09.169964 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:09.169938 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-776cc66d47-zf2dd" event={"ID":"ef60d381-c326-449b-a5f5-47b539535199","Type":"ContainerStarted","Data":"5f8ae2a2d376d9fe512135d3d13187eae57651aa74aa3a396e0bc602f9bbcbe4"} Oct 30 18:45:09.170063 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:09.169995 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-776cc66d47-zf2dd" Oct 30 18:45:09.171481 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:09.171461 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256" event={"ID":"ee74582d-1edf-412c-9a76-459d08a10cc3","Type":"ContainerStarted","Data":"3936573f2d464ae43e0cbe7e95c6a198a42054f70afdadc04efd888cb206eccf"} Oct 30 18:45:09.171600 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:09.171511 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256" Oct 30 18:45:09.320349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:09.320291 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/model-serving-api-75bd89fdfc-b2qh7" podStartSLOduration=3.026568937 podStartE2EDuration="5.320276752s" totalImagesPullingTime="2.293707815s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:09.318892422 +0000 UTC m=+379.809020257" watchObservedRunningTime="2026-10-30 18:45:09.320276752 +0000 UTC m=+379.810404563" Oct 30 18:45:09.431805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:09.431751 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256" podStartSLOduration=2.651467487 podStartE2EDuration="5.431733486s" totalImagesPullingTime="2.780265999s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:09.431405896 +0000 UTC m=+379.921533716" watchObservedRunningTime="2026-10-30 18:45:09.431733486 +0000 UTC m=+379.921861310" Oct 30 18:45:09.545419 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:09.545363 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-776cc66d47-zf2dd" podStartSLOduration=2.019959215 podStartE2EDuration="5.545348075s" totalImagesPullingTime="3.52538886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:45:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:45:09.544138694 +0000 UTC m=+380.034266515" watchObservedRunningTime="2026-10-30 18:45:09.545348075 +0000 UTC m=+380.035475894" Oct 30 18:45:11.015274 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:11.015238 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:11.239911 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:11.239879 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:11.407871 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:11.407840 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:11.579382 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:11.579347 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:11.742590 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:11.742502 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:11.907056 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:11.907021 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:12.078124 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:12.078040 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:12.259884 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:12.259839 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:13.725795 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:13.725762 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:45:20.116780 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:45:20.116735 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:45:20.180431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:20.180402 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-75bd89fdfc-b2qh7" Oct 30 18:45:40.178463 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:40.178430 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256" Oct 30 18:45:40.181367 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:45:40.181345 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-776cc66d47-zf2dd" Oct 30 18:45:50.116893 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:45:50.116805 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:46:12.190978 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:12.190943 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:12.379934 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:12.379903 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:13.842702 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:13.842670 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:14.031644 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:14.031609 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:14.215556 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:14.215522 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:20.116376 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:46:20.116344 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:46:29.533695 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:29.533659 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:30.322901 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:30.322870 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:31.130721 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:31.130638 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:33.367011 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:33.366977 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:33.551515 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:33.551480 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:34.715992 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:34.715961 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:35.985784 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:35.985755 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf"] Oct 30 18:46:35.989040 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:35.989023 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf" Oct 30 18:46:36.096515 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:36.096487 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Oct 30 18:46:36.096515 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:36.096507 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Oct 30 18:46:36.096754 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:36.096631 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-f48m2\"" Oct 30 18:46:36.097444 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:36.097426 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf"] Oct 30 18:46:36.115475 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:36.115437 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mg5xt\" (UniqueName: \"kubernetes.io/projected/4ad66dcf-cbf0-434c-a261-1629cccf5c09-kube-api-access-mg5xt\") pod \"limitador-operator-controller-manager-856dd584c5-tbqrf\" (UID: \"4ad66dcf-cbf0-434c-a261-1629cccf5c09\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf" Oct 30 18:46:36.216228 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:36.216200 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mg5xt\" (UniqueName: \"kubernetes.io/projected/4ad66dcf-cbf0-434c-a261-1629cccf5c09-kube-api-access-mg5xt\") pod \"limitador-operator-controller-manager-856dd584c5-tbqrf\" (UID: \"4ad66dcf-cbf0-434c-a261-1629cccf5c09\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf" Oct 30 18:46:36.277241 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:36.277178 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mg5xt\" (UniqueName: \"kubernetes.io/projected/4ad66dcf-cbf0-434c-a261-1629cccf5c09-kube-api-access-mg5xt\") pod \"limitador-operator-controller-manager-856dd584c5-tbqrf\" (UID: \"4ad66dcf-cbf0-434c-a261-1629cccf5c09\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf" Oct 30 18:46:36.301817 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:36.301795 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf" Oct 30 18:46:36.364111 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:36.364034 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:36.566395 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:36.566326 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:36.668236 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:36.668206 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf"] Oct 30 18:46:36.672101 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:46:36.672073 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ad66dcf_cbf0_434c_a261_1629cccf5c09.slice/crio-de4f198aba3804aab5abfab6525394d63231ca17cca82dce56cf518983ec4a6a WatchSource:0}: Error finding container de4f198aba3804aab5abfab6525394d63231ca17cca82dce56cf518983ec4a6a: Status 404 returned error can't find the container with id de4f198aba3804aab5abfab6525394d63231ca17cca82dce56cf518983ec4a6a Oct 30 18:46:36.760627 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:36.760592 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:37.554405 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:37.554372 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf" event={"ID":"4ad66dcf-cbf0-434c-a261-1629cccf5c09","Type":"ContainerStarted","Data":"de4f198aba3804aab5abfab6525394d63231ca17cca82dce56cf518983ec4a6a"} Oct 30 18:46:38.779436 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:38.779401 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:38.968853 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:38.968814 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:39.543586 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:39.543531 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:40.164670 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:40.164629 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:40.572104 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:40.572066 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf" event={"ID":"4ad66dcf-cbf0-434c-a261-1629cccf5c09","Type":"ContainerStarted","Data":"49ac8a3b8d483e331cc107752d6d74c7247d151db9532d902d31bcba4f4f8033"} Oct 30 18:46:40.572283 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:40.572152 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf" Oct 30 18:46:40.688442 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:40.688346 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf" podStartSLOduration=1.941769157 podStartE2EDuration="5.688332871s" totalImagesPullingTime="3.746563714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:40.687636842 +0000 UTC m=+471.177764659" watchObservedRunningTime="2026-10-30 18:46:40.688332871 +0000 UTC m=+471.178460691" Oct 30 18:46:41.171800 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:41.171767 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-twqpm"] Oct 30 18:46:41.175227 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:41.175211 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-twqpm" Oct 30 18:46:41.232375 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:41.232340 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Oct 30 18:46:41.232518 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:41.232443 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-m4z6x\"" Oct 30 18:46:41.261383 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:41.261353 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n5v5\" (UniqueName: \"kubernetes.io/projected/c9b5f0b2-375b-423f-9a97-6c797ca45ab7-kube-api-access-4n5v5\") pod \"dns-operator-controller-manager-65bfd79598-twqpm\" (UID: \"c9b5f0b2-375b-423f-9a97-6c797ca45ab7\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-twqpm" Oct 30 18:46:41.283247 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:41.283212 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-twqpm"] Oct 30 18:46:41.361845 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:41.361809 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4n5v5\" (UniqueName: \"kubernetes.io/projected/c9b5f0b2-375b-423f-9a97-6c797ca45ab7-kube-api-access-4n5v5\") pod \"dns-operator-controller-manager-65bfd79598-twqpm\" (UID: \"c9b5f0b2-375b-423f-9a97-6c797ca45ab7\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-twqpm" Oct 30 18:46:41.419592 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:41.419548 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n5v5\" (UniqueName: \"kubernetes.io/projected/c9b5f0b2-375b-423f-9a97-6c797ca45ab7-kube-api-access-4n5v5\") pod \"dns-operator-controller-manager-65bfd79598-twqpm\" (UID: \"c9b5f0b2-375b-423f-9a97-6c797ca45ab7\") " pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-twqpm" Oct 30 18:46:41.487634 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:41.487547 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-twqpm" Oct 30 18:46:41.860445 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:41.860368 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-65bfd79598-twqpm"] Oct 30 18:46:41.890216 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:46:41.890185 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9b5f0b2_375b_423f_9a97_6c797ca45ab7.slice/crio-83fc63aecc4ed38389c63bc0e7f9f3f721545b58026ff4127e3428a682b0a490 WatchSource:0}: Error finding container 83fc63aecc4ed38389c63bc0e7f9f3f721545b58026ff4127e3428a682b0a490: Status 404 returned error can't find the container with id 83fc63aecc4ed38389c63bc0e7f9f3f721545b58026ff4127e3428a682b0a490 Oct 30 18:46:42.583168 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:42.583134 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-twqpm" event={"ID":"c9b5f0b2-375b-423f-9a97-6c797ca45ab7","Type":"ContainerStarted","Data":"83fc63aecc4ed38389c63bc0e7f9f3f721545b58026ff4127e3428a682b0a490"} Oct 30 18:46:44.532712 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:44.532684 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:44.600388 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:44.600355 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-twqpm" event={"ID":"c9b5f0b2-375b-423f-9a97-6c797ca45ab7","Type":"ContainerStarted","Data":"4c01edc57f80279782536ca8b46ec6f453617ff3a6503873de8f3d1308fcdb4a"} Oct 30 18:46:44.600497 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:44.600441 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-twqpm" Oct 30 18:46:44.708322 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:44.708290 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:44.722554 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:44.722505 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-twqpm" podStartSLOduration=1.116461529 podStartE2EDuration="3.722491001s" totalImagesPullingTime="2.606029472s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:44.721591081 +0000 UTC m=+475.211718894" watchObservedRunningTime="2026-10-30 18:46:44.722491001 +0000 UTC m=+475.212618815" Oct 30 18:46:44.936481 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:44.936452 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:45.114600 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:45.114547 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:45.330556 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:45.330444 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:45.514310 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:45.514276 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:45.751211 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:45.751157 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:46.068700 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:46.068624 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:50.116986 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:46:50.116949 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:46:51.552289 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:51.552246 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:51.579384 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:51.579357 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf" Oct 30 18:46:51.734106 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:51.734069 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:53.232422 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:53.232389 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q6rn9"] Oct 30 18:46:53.235871 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:53.235853 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q6rn9" Oct 30 18:46:53.291987 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:53.291958 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q6rn9"] Oct 30 18:46:53.340741 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:53.340713 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-l677f\"" Oct 30 18:46:53.341853 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:53.341829 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q6rn9"] Oct 30 18:46:53.342074 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:46:53.342053 5894 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-v84dp], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q6rn9" podUID="739e6091-37f5-4dce-8155-6fe2c80c6361" Oct 30 18:46:53.363194 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:53.363168 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/739e6091-37f5-4dce-8155-6fe2c80c6361-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b76784d48-q6rn9\" (UID: \"739e6091-37f5-4dce-8155-6fe2c80c6361\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q6rn9" Oct 30 18:46:53.363293 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:53.363223 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v84dp\" (UniqueName: \"kubernetes.io/projected/739e6091-37f5-4dce-8155-6fe2c80c6361-kube-api-access-v84dp\") pod \"kuadrant-operator-controller-manager-6b76784d48-q6rn9\" (UID: \"739e6091-37f5-4dce-8155-6fe2c80c6361\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q6rn9" Oct 30 18:46:53.464444 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:53.464414 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/739e6091-37f5-4dce-8155-6fe2c80c6361-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b76784d48-q6rn9\" (UID: \"739e6091-37f5-4dce-8155-6fe2c80c6361\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q6rn9" Oct 30 18:46:53.464620 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:53.464488 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v84dp\" (UniqueName: \"kubernetes.io/projected/739e6091-37f5-4dce-8155-6fe2c80c6361-kube-api-access-v84dp\") pod \"kuadrant-operator-controller-manager-6b76784d48-q6rn9\" (UID: \"739e6091-37f5-4dce-8155-6fe2c80c6361\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q6rn9" Oct 30 18:46:53.464827 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:53.464805 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/739e6091-37f5-4dce-8155-6fe2c80c6361-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6b76784d48-q6rn9\" (UID: \"739e6091-37f5-4dce-8155-6fe2c80c6361\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q6rn9" Oct 30 18:46:53.521835 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:53.521768 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v84dp\" (UniqueName: \"kubernetes.io/projected/739e6091-37f5-4dce-8155-6fe2c80c6361-kube-api-access-v84dp\") pod \"kuadrant-operator-controller-manager-6b76784d48-q6rn9\" (UID: \"739e6091-37f5-4dce-8155-6fe2c80c6361\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q6rn9" Oct 30 18:46:53.639527 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:53.639497 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q6rn9" Oct 30 18:46:53.646724 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:53.646698 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q6rn9" Oct 30 18:46:53.766989 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:53.766959 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/739e6091-37f5-4dce-8155-6fe2c80c6361-extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/739e6091-37f5-4dce-8155-6fe2c80c6361-extensions-socket-volume\") pod \"739e6091-37f5-4dce-8155-6fe2c80c6361\" (UID: \"739e6091-37f5-4dce-8155-6fe2c80c6361\") " Oct 30 18:46:53.766989 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:53.766991 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/739e6091-37f5-4dce-8155-6fe2c80c6361-kube-api-access-v84dp\" (UniqueName: \"kubernetes.io/projected/739e6091-37f5-4dce-8155-6fe2c80c6361-kube-api-access-v84dp\") pod \"739e6091-37f5-4dce-8155-6fe2c80c6361\" (UID: \"739e6091-37f5-4dce-8155-6fe2c80c6361\") " Oct 30 18:46:53.767275 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:53.767251 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/739e6091-37f5-4dce-8155-6fe2c80c6361-extensions-socket-volume" pod "739e6091-37f5-4dce-8155-6fe2c80c6361" (UID: "739e6091-37f5-4dce-8155-6fe2c80c6361"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:46:53.769055 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:53.769030 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/739e6091-37f5-4dce-8155-6fe2c80c6361-kube-api-access-v84dp" pod "739e6091-37f5-4dce-8155-6fe2c80c6361" (UID: "739e6091-37f5-4dce-8155-6fe2c80c6361"). InnerVolumeSpecName "kube-api-access-v84dp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:53.868067 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:53.868040 5894 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/739e6091-37f5-4dce-8155-6fe2c80c6361-extensions-socket-volume\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:46:53.868067 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:53.868064 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v84dp\" (UniqueName: \"kubernetes.io/projected/739e6091-37f5-4dce-8155-6fe2c80c6361-kube-api-access-v84dp\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:46:54.644263 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:54.644229 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q6rn9" Oct 30 18:46:54.871417 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:54.871387 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q6rn9"] Oct 30 18:46:54.874815 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:54.874790 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6b76784d48-q6rn9"] Oct 30 18:46:55.144168 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.144138 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf"] Oct 30 18:46:55.144415 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.144390 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf" podUID="4ad66dcf-cbf0-434c-a261-1629cccf5c09" containerName="manager" containerID="cri-o://49ac8a3b8d483e331cc107752d6d74c7247d151db9532d902d31bcba4f4f8033" gracePeriod=2 Oct 30 18:46:55.161785 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.161755 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf"] Oct 30 18:46:55.175314 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.175289 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-8hzrm"] Oct 30 18:46:55.175808 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.175785 5894 cpu_manager.go:414] "containerMap: removing container" podUID="4ad66dcf-cbf0-434c-a261-1629cccf5c09" containerName="manager" Oct 30 18:46:55.175808 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.175809 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4ad66dcf-cbf0-434c-a261-1629cccf5c09" containerName="manager" Oct 30 18:46:55.175958 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.175881 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="4ad66dcf-cbf0-434c-a261-1629cccf5c09" containerName="manager" Oct 30 18:46:55.179045 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.179026 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-8hzrm" Oct 30 18:46:55.195416 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:46:55.195388 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-856dd584c5-tbqrf\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="4ad66dcf-cbf0-434c-a261-1629cccf5c09" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf" Oct 30 18:46:55.237722 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.237695 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-8hzrm"] Oct 30 18:46:55.281969 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.281937 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4jtq\" (UniqueName: \"kubernetes.io/projected/c4f74804-9c53-4f89-b0e7-50599bb10a9f-kube-api-access-w4jtq\") pod \"limitador-operator-controller-manager-856dd584c5-8hzrm\" (UID: \"c4f74804-9c53-4f89-b0e7-50599bb10a9f\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-8hzrm" Oct 30 18:46:55.358676 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:46:55.358637 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-856dd584c5-tbqrf\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="4ad66dcf-cbf0-434c-a261-1629cccf5c09" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf" Oct 30 18:46:55.383097 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.383066 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4jtq\" (UniqueName: \"kubernetes.io/projected/c4f74804-9c53-4f89-b0e7-50599bb10a9f-kube-api-access-w4jtq\") pod \"limitador-operator-controller-manager-856dd584c5-8hzrm\" (UID: \"c4f74804-9c53-4f89-b0e7-50599bb10a9f\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-8hzrm" Oct 30 18:46:55.429731 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.429709 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf" Oct 30 18:46:55.441037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.441007 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4jtq\" (UniqueName: \"kubernetes.io/projected/c4f74804-9c53-4f89-b0e7-50599bb10a9f-kube-api-access-w4jtq\") pod \"limitador-operator-controller-manager-856dd584c5-8hzrm\" (UID: \"c4f74804-9c53-4f89-b0e7-50599bb10a9f\") " pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-8hzrm" Oct 30 18:46:55.480761 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:46:55.480731 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-856dd584c5-tbqrf\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="4ad66dcf-cbf0-434c-a261-1629cccf5c09" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf" Oct 30 18:46:55.524028 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.524007 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-8hzrm" Oct 30 18:46:55.585076 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.585048 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4ad66dcf-cbf0-434c-a261-1629cccf5c09-kube-api-access-mg5xt\" (UniqueName: \"kubernetes.io/projected/4ad66dcf-cbf0-434c-a261-1629cccf5c09-kube-api-access-mg5xt\") pod \"4ad66dcf-cbf0-434c-a261-1629cccf5c09\" (UID: \"4ad66dcf-cbf0-434c-a261-1629cccf5c09\") " Oct 30 18:46:55.587792 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.587766 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ad66dcf-cbf0-434c-a261-1629cccf5c09-kube-api-access-mg5xt" pod "4ad66dcf-cbf0-434c-a261-1629cccf5c09" (UID: "4ad66dcf-cbf0-434c-a261-1629cccf5c09"). InnerVolumeSpecName "kube-api-access-mg5xt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:46:55.606683 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.606661 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-65bfd79598-twqpm" Oct 30 18:46:55.648378 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.648347 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4ad66dcf-cbf0-434c-a261-1629cccf5c09" path="/var/lib/kubelet/pods/4ad66dcf-cbf0-434c-a261-1629cccf5c09/volumes" Oct 30 18:46:55.648826 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.648809 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="739e6091-37f5-4dce-8155-6fe2c80c6361" path="/var/lib/kubelet/pods/739e6091-37f5-4dce-8155-6fe2c80c6361/volumes" Oct 30 18:46:55.649725 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.649702 5894 generic.go:356] "Generic (PLEG): container finished" podID="4ad66dcf-cbf0-434c-a261-1629cccf5c09" containerID="49ac8a3b8d483e331cc107752d6d74c7247d151db9532d902d31bcba4f4f8033" exitCode=0 Oct 30 18:46:55.649803 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.649742 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf" Oct 30 18:46:55.649803 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.649796 5894 scope.go:122] "RemoveContainer" containerID="49ac8a3b8d483e331cc107752d6d74c7247d151db9532d902d31bcba4f4f8033" Oct 30 18:46:55.657715 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:46:55.657689 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-856dd584c5-tbqrf\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="4ad66dcf-cbf0-434c-a261-1629cccf5c09" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-tbqrf" Oct 30 18:46:55.658198 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.658181 5894 scope.go:122] "RemoveContainer" containerID="49ac8a3b8d483e331cc107752d6d74c7247d151db9532d902d31bcba4f4f8033" Oct 30 18:46:55.658497 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:46:55.658479 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"49ac8a3b8d483e331cc107752d6d74c7247d151db9532d902d31bcba4f4f8033\": container with ID starting with 49ac8a3b8d483e331cc107752d6d74c7247d151db9532d902d31bcba4f4f8033 not found: ID does not exist" containerID="49ac8a3b8d483e331cc107752d6d74c7247d151db9532d902d31bcba4f4f8033" Oct 30 18:46:55.658569 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.658507 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"49ac8a3b8d483e331cc107752d6d74c7247d151db9532d902d31bcba4f4f8033"} err="failed to get container status \"49ac8a3b8d483e331cc107752d6d74c7247d151db9532d902d31bcba4f4f8033\": rpc error: code = NotFound desc = could not find container \"49ac8a3b8d483e331cc107752d6d74c7247d151db9532d902d31bcba4f4f8033\": container with ID starting with 49ac8a3b8d483e331cc107752d6d74c7247d151db9532d902d31bcba4f4f8033 not found: ID does not exist" Oct 30 18:46:55.685889 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.685844 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mg5xt\" (UniqueName: \"kubernetes.io/projected/4ad66dcf-cbf0-434c-a261-1629cccf5c09-kube-api-access-mg5xt\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:46:55.905597 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:55.905399 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-856dd584c5-8hzrm"] Oct 30 18:46:55.908279 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:46:55.908253 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4f74804_9c53_4f89_b0e7_50599bb10a9f.slice/crio-691ede27708b9dba66adc28af2ad6bafb10ec96f4baad3daf6e7763ef7f888bc WatchSource:0}: Error finding container 691ede27708b9dba66adc28af2ad6bafb10ec96f4baad3daf6e7763ef7f888bc: Status 404 returned error can't find the container with id 691ede27708b9dba66adc28af2ad6bafb10ec96f4baad3daf6e7763ef7f888bc Oct 30 18:46:56.384444 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:56.384361 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:56.595407 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:56.595374 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:56.656341 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:56.656312 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-8hzrm" event={"ID":"c4f74804-9c53-4f89-b0e7-50599bb10a9f","Type":"ContainerStarted","Data":"9a26272bcfbb9d5c0d67f4853c1b21d33498296d5ebca36641d7961b0366cbfc"} Oct 30 18:46:56.656754 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:56.656360 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-8hzrm" event={"ID":"c4f74804-9c53-4f89-b0e7-50599bb10a9f","Type":"ContainerStarted","Data":"691ede27708b9dba66adc28af2ad6bafb10ec96f4baad3daf6e7763ef7f888bc"} Oct 30 18:46:56.656754 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:56.656457 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-8hzrm" Oct 30 18:46:56.771638 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:56.771565 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-8hzrm" podStartSLOduration=1.771550006 podStartE2EDuration="1.771550006s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:46:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:46:56.771162506 +0000 UTC m=+487.261290327" watchObservedRunningTime="2026-10-30 18:46:56.771550006 +0000 UTC m=+487.261677826" Oct 30 18:46:58.804744 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:58.804681 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:46:59.030022 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:46:59.029988 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:05.743500 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:05.743460 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:05.935953 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:05.935919 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:06.162916 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:06.162885 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:06.346339 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:06.346308 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:06.539599 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:06.539477 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:07.662175 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:07.662147 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-856dd584c5-8hzrm" Oct 30 18:47:13.360184 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:13.360153 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:13.535358 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:13.535256 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:14.399308 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:14.399273 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:18.152482 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:18.152443 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:18.341621 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:18.341567 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:20.116304 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:47:20.116264 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:47:20.616802 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:20.616769 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:20.801006 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:20.800973 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:25.466084 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:25.466049 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:25.663077 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:25.663038 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:30.408729 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:30.408692 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:42.286785 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:42.286745 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:42.327302 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:42.327273 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hd5vv"] Oct 30 18:47:42.330485 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:42.330468 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-hd5vv" Oct 30 18:47:42.382242 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:42.382215 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Oct 30 18:47:42.432154 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:42.432125 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-hdxfm\"" Oct 30 18:47:42.433385 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:42.433365 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hd5vv"] Oct 30 18:47:42.468321 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:42.468293 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p22kl\" (UniqueName: \"kubernetes.io/projected/13a94c55-f8f3-45a6-9d86-972e7c22b0c7-kube-api-access-p22kl\") pod \"limitador-limitador-74dfcfb894-hd5vv\" (UID: \"13a94c55-f8f3-45a6-9d86-972e7c22b0c7\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hd5vv" Oct 30 18:47:42.468433 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:42.468341 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/13a94c55-f8f3-45a6-9d86-972e7c22b0c7-config-file\") pod \"limitador-limitador-74dfcfb894-hd5vv\" (UID: \"13a94c55-f8f3-45a6-9d86-972e7c22b0c7\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hd5vv" Oct 30 18:47:42.568212 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:42.568147 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hd5vv"] Oct 30 18:47:42.569216 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:42.569194 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p22kl\" (UniqueName: \"kubernetes.io/projected/13a94c55-f8f3-45a6-9d86-972e7c22b0c7-kube-api-access-p22kl\") pod \"limitador-limitador-74dfcfb894-hd5vv\" (UID: \"13a94c55-f8f3-45a6-9d86-972e7c22b0c7\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hd5vv" Oct 30 18:47:42.569343 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:42.569253 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/13a94c55-f8f3-45a6-9d86-972e7c22b0c7-config-file\") pod \"limitador-limitador-74dfcfb894-hd5vv\" (UID: \"13a94c55-f8f3-45a6-9d86-972e7c22b0c7\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hd5vv" Oct 30 18:47:42.569891 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:42.569870 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/13a94c55-f8f3-45a6-9d86-972e7c22b0c7-config-file\") pod \"limitador-limitador-74dfcfb894-hd5vv\" (UID: \"13a94c55-f8f3-45a6-9d86-972e7c22b0c7\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hd5vv" Oct 30 18:47:42.630739 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:42.630708 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p22kl\" (UniqueName: \"kubernetes.io/projected/13a94c55-f8f3-45a6-9d86-972e7c22b0c7-kube-api-access-p22kl\") pod \"limitador-limitador-74dfcfb894-hd5vv\" (UID: \"13a94c55-f8f3-45a6-9d86-972e7c22b0c7\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-hd5vv" Oct 30 18:47:42.642418 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:42.642394 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-hd5vv" Oct 30 18:47:42.649434 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:42.649410 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:42.816061 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:42.816023 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:43.011263 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:43.011231 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hd5vv"] Oct 30 18:47:43.014361 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:47:43.014334 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13a94c55_f8f3_45a6_9d86_972e7c22b0c7.slice/crio-08c73b78f9d3c486841b27a9888f3403cd7dfcccf9c5a22ec0f1ab87b24f5d2a WatchSource:0}: Error finding container 08c73b78f9d3c486841b27a9888f3403cd7dfcccf9c5a22ec0f1ab87b24f5d2a: Status 404 returned error can't find the container with id 08c73b78f9d3c486841b27a9888f3403cd7dfcccf9c5a22ec0f1ab87b24f5d2a Oct 30 18:47:43.858885 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:43.858853 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-hd5vv" event={"ID":"13a94c55-f8f3-45a6-9d86-972e7c22b0c7","Type":"ContainerStarted","Data":"08c73b78f9d3c486841b27a9888f3403cd7dfcccf9c5a22ec0f1ab87b24f5d2a"} Oct 30 18:47:43.989942 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:43.989893 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:44.169824 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:44.169792 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:45.867915 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:45.867870 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-hd5vv" event={"ID":"13a94c55-f8f3-45a6-9d86-972e7c22b0c7","Type":"ContainerStarted","Data":"6b1b7fe33eb2a22e1c44f7a4be06824f5677cbdc7e55ddd36fad96179819000a"} Oct 30 18:47:45.868284 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:45.868009 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-hd5vv" Oct 30 18:47:45.981179 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:45.981118 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-hd5vv" podStartSLOduration=1.6716869170000002 podStartE2EDuration="3.98110051s" totalImagesPullingTime="2.309413593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:47:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:47:45.980387571 +0000 UTC m=+536.470515394" watchObservedRunningTime="2026-10-30 18:47:45.98110051 +0000 UTC m=+536.471228341" Oct 30 18:47:46.242386 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:46.242317 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:46.405599 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:46.405547 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:47:50.116892 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:47:50.116812 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:47:56.873451 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:47:56.873422 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-hd5vv" Oct 30 18:48:00.854352 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:00.854313 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hd5vv"] Oct 30 18:48:00.854805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:00.854511 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-hd5vv" podUID="13a94c55-f8f3-45a6-9d86-972e7c22b0c7" containerName="limitador" containerID="cri-o://6b1b7fe33eb2a22e1c44f7a4be06824f5677cbdc7e55ddd36fad96179819000a" gracePeriod=30 Oct 30 18:48:01.399547 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:01.399522 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-hd5vv" Oct 30 18:48:01.421300 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:01.421236 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/13a94c55-f8f3-45a6-9d86-972e7c22b0c7-config-file\" (UniqueName: \"kubernetes.io/configmap/13a94c55-f8f3-45a6-9d86-972e7c22b0c7-config-file\") pod \"13a94c55-f8f3-45a6-9d86-972e7c22b0c7\" (UID: \"13a94c55-f8f3-45a6-9d86-972e7c22b0c7\") " Oct 30 18:48:01.421419 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:01.421325 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/13a94c55-f8f3-45a6-9d86-972e7c22b0c7-kube-api-access-p22kl\" (UniqueName: \"kubernetes.io/projected/13a94c55-f8f3-45a6-9d86-972e7c22b0c7-kube-api-access-p22kl\") pod \"13a94c55-f8f3-45a6-9d86-972e7c22b0c7\" (UID: \"13a94c55-f8f3-45a6-9d86-972e7c22b0c7\") " Oct 30 18:48:01.421654 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:01.421633 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13a94c55-f8f3-45a6-9d86-972e7c22b0c7-config-file" pod "13a94c55-f8f3-45a6-9d86-972e7c22b0c7" (UID: "13a94c55-f8f3-45a6-9d86-972e7c22b0c7"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:48:01.423274 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:01.423250 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13a94c55-f8f3-45a6-9d86-972e7c22b0c7-kube-api-access-p22kl" pod "13a94c55-f8f3-45a6-9d86-972e7c22b0c7" (UID: "13a94c55-f8f3-45a6-9d86-972e7c22b0c7"). InnerVolumeSpecName "kube-api-access-p22kl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:48:01.522509 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:01.522477 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p22kl\" (UniqueName: \"kubernetes.io/projected/13a94c55-f8f3-45a6-9d86-972e7c22b0c7-kube-api-access-p22kl\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:48:01.522509 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:01.522504 5894 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/13a94c55-f8f3-45a6-9d86-972e7c22b0c7-config-file\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:48:01.937040 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:01.937005 5894 generic.go:356] "Generic (PLEG): container finished" podID="13a94c55-f8f3-45a6-9d86-972e7c22b0c7" containerID="6b1b7fe33eb2a22e1c44f7a4be06824f5677cbdc7e55ddd36fad96179819000a" exitCode=0 Oct 30 18:48:01.937468 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:01.937053 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-hd5vv" Oct 30 18:48:01.937468 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:01.937093 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-hd5vv" event={"ID":"13a94c55-f8f3-45a6-9d86-972e7c22b0c7","Type":"ContainerDied","Data":"6b1b7fe33eb2a22e1c44f7a4be06824f5677cbdc7e55ddd36fad96179819000a"} Oct 30 18:48:01.937468 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:01.937152 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-hd5vv" event={"ID":"13a94c55-f8f3-45a6-9d86-972e7c22b0c7","Type":"ContainerDied","Data":"08c73b78f9d3c486841b27a9888f3403cd7dfcccf9c5a22ec0f1ab87b24f5d2a"} Oct 30 18:48:01.937468 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:01.937181 5894 scope.go:122] "RemoveContainer" containerID="6b1b7fe33eb2a22e1c44f7a4be06824f5677cbdc7e55ddd36fad96179819000a" Oct 30 18:48:01.945464 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:01.945443 5894 scope.go:122] "RemoveContainer" containerID="6b1b7fe33eb2a22e1c44f7a4be06824f5677cbdc7e55ddd36fad96179819000a" Oct 30 18:48:01.945751 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:48:01.945731 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b1b7fe33eb2a22e1c44f7a4be06824f5677cbdc7e55ddd36fad96179819000a\": container with ID starting with 6b1b7fe33eb2a22e1c44f7a4be06824f5677cbdc7e55ddd36fad96179819000a not found: ID does not exist" containerID="6b1b7fe33eb2a22e1c44f7a4be06824f5677cbdc7e55ddd36fad96179819000a" Oct 30 18:48:01.945811 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:01.945761 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b1b7fe33eb2a22e1c44f7a4be06824f5677cbdc7e55ddd36fad96179819000a"} err="failed to get container status \"6b1b7fe33eb2a22e1c44f7a4be06824f5677cbdc7e55ddd36fad96179819000a\": rpc error: code = NotFound desc = could not find container \"6b1b7fe33eb2a22e1c44f7a4be06824f5677cbdc7e55ddd36fad96179819000a\": container with ID starting with 6b1b7fe33eb2a22e1c44f7a4be06824f5677cbdc7e55ddd36fad96179819000a not found: ID does not exist" Oct 30 18:48:02.100248 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:02.100215 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hd5vv"] Oct 30 18:48:02.149900 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:02.149870 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-hd5vv"] Oct 30 18:48:03.647756 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:03.647724 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="13a94c55-f8f3-45a6-9d86-972e7c22b0c7" path="/var/lib/kubelet/pods/13a94c55-f8f3-45a6-9d86-972e7c22b0c7/volumes" Oct 30 18:48:05.513679 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:05.513646 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:05.699984 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:05.699954 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:06.160931 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.160900 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-nmj5m"] Oct 30 18:48:06.161266 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.161253 5894 cpu_manager.go:414] "containerMap: removing container" podUID="13a94c55-f8f3-45a6-9d86-972e7c22b0c7" containerName="limitador" Oct 30 18:48:06.161313 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.161268 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="13a94c55-f8f3-45a6-9d86-972e7c22b0c7" containerName="limitador" Oct 30 18:48:06.161347 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.161336 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="13a94c55-f8f3-45a6-9d86-972e7c22b0c7" containerName="limitador" Oct 30 18:48:06.164468 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.164453 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nmj5m" Oct 30 18:48:06.218021 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.217996 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Oct 30 18:48:06.218176 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.218057 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Oct 30 18:48:06.261775 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.261742 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4ecfa4cd-fb2a-43b0-9376-8280e2b3e154-data\") pod \"postgres-7cdbf6dc44-nmj5m\" (UID: \"4ecfa4cd-fb2a-43b0-9376-8280e2b3e154\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nmj5m" Oct 30 18:48:06.261775 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.261776 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sb526\" (UniqueName: \"kubernetes.io/projected/4ecfa4cd-fb2a-43b0-9376-8280e2b3e154-kube-api-access-sb526\") pod \"postgres-7cdbf6dc44-nmj5m\" (UID: \"4ecfa4cd-fb2a-43b0-9376-8280e2b3e154\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nmj5m" Oct 30 18:48:06.268792 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.268768 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-nmj5m"] Oct 30 18:48:06.284911 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.284889 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:06.285706 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.285684 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Oct 30 18:48:06.285706 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.285697 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-9jksw\"" Oct 30 18:48:06.362535 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.362505 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4ecfa4cd-fb2a-43b0-9376-8280e2b3e154-data\") pod \"postgres-7cdbf6dc44-nmj5m\" (UID: \"4ecfa4cd-fb2a-43b0-9376-8280e2b3e154\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nmj5m" Oct 30 18:48:06.362535 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.362536 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sb526\" (UniqueName: \"kubernetes.io/projected/4ecfa4cd-fb2a-43b0-9376-8280e2b3e154-kube-api-access-sb526\") pod \"postgres-7cdbf6dc44-nmj5m\" (UID: \"4ecfa4cd-fb2a-43b0-9376-8280e2b3e154\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nmj5m" Oct 30 18:48:06.362898 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.362877 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4ecfa4cd-fb2a-43b0-9376-8280e2b3e154-data\") pod \"postgres-7cdbf6dc44-nmj5m\" (UID: \"4ecfa4cd-fb2a-43b0-9376-8280e2b3e154\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nmj5m" Oct 30 18:48:06.418971 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.418916 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sb526\" (UniqueName: \"kubernetes.io/projected/4ecfa4cd-fb2a-43b0-9376-8280e2b3e154-kube-api-access-sb526\") pod \"postgres-7cdbf6dc44-nmj5m\" (UID: \"4ecfa4cd-fb2a-43b0-9376-8280e2b3e154\") " pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nmj5m" Oct 30 18:48:06.476670 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.476644 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nmj5m" Oct 30 18:48:06.843132 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.843057 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-7cdbf6dc44-nmj5m"] Oct 30 18:48:06.846282 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:48:06.846246 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ecfa4cd_fb2a_43b0_9376_8280e2b3e154.slice/crio-1c7e45b885cd41f9a02609cd7662af9415b752d3697a506b87de7b6547fb2587 WatchSource:0}: Error finding container 1c7e45b885cd41f9a02609cd7662af9415b752d3697a506b87de7b6547fb2587: Status 404 returned error can't find the container with id 1c7e45b885cd41f9a02609cd7662af9415b752d3697a506b87de7b6547fb2587 Oct 30 18:48:06.960898 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:06.960857 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nmj5m" event={"ID":"4ecfa4cd-fb2a-43b0-9376-8280e2b3e154","Type":"ContainerStarted","Data":"1c7e45b885cd41f9a02609cd7662af9415b752d3697a506b87de7b6547fb2587"} Oct 30 18:48:09.883536 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:09.883498 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:13.995607 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:13.995544 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nmj5m" event={"ID":"4ecfa4cd-fb2a-43b0-9376-8280e2b3e154","Type":"ContainerStarted","Data":"678bc4d6d5e0ee1ec9cdce639d89829dc0cef495a707a1be91124ce32de31b5d"} Oct 30 18:48:13.995977 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:13.995635 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nmj5m" Oct 30 18:48:14.736349 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:14.736318 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:19.491800 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:19.491768 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:20.028773 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:20.028746 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nmj5m" Oct 30 18:48:20.115896 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:48:20.115861 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:48:20.142142 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:20.142084 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-7cdbf6dc44-nmj5m" podStartSLOduration=7.717856144 podStartE2EDuration="14.142070914s" totalImagesPullingTime="6.42421477s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:14.111620914 +0000 UTC m=+564.601748735" watchObservedRunningTime="2026-10-30 18:48:20.142070914 +0000 UTC m=+570.632198734" Oct 30 18:48:20.579400 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:20.579368 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:20.871340 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:20.871258 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:31.423181 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.423140 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:31.447313 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.447284 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-577f596567-xwt65"] Oct 30 18:48:31.451880 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.451863 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-xwt65" Oct 30 18:48:31.506007 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.505976 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\"" Oct 30 18:48:31.506249 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.506232 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wwn5x\"" Oct 30 18:48:31.506320 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.506305 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Oct 30 18:48:31.555982 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.555954 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Oct 30 18:48:31.557109 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.557089 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577f596567-xwt65"] Oct 30 18:48:31.582046 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.582022 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01ec44aa-a128-48a4-9d58-0f643487f5ab-metrics-certs\") pod \"maas-controller-577f596567-xwt65\" (UID: \"01ec44aa-a128-48a4-9d58-0f643487f5ab\") " pod="opendatahub/maas-controller-577f596567-xwt65" Oct 30 18:48:31.582192 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.582067 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpdsj\" (UniqueName: \"kubernetes.io/projected/01ec44aa-a128-48a4-9d58-0f643487f5ab-kube-api-access-zpdsj\") pod \"maas-controller-577f596567-xwt65\" (UID: \"01ec44aa-a128-48a4-9d58-0f643487f5ab\") " pod="opendatahub/maas-controller-577f596567-xwt65" Oct 30 18:48:31.582192 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.582090 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01ec44aa-a128-48a4-9d58-0f643487f5ab-webhook-cert\") pod \"maas-controller-577f596567-xwt65\" (UID: \"01ec44aa-a128-48a4-9d58-0f643487f5ab\") " pod="opendatahub/maas-controller-577f596567-xwt65" Oct 30 18:48:31.597450 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.597423 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:31.683425 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.683342 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01ec44aa-a128-48a4-9d58-0f643487f5ab-metrics-certs\") pod \"maas-controller-577f596567-xwt65\" (UID: \"01ec44aa-a128-48a4-9d58-0f643487f5ab\") " pod="opendatahub/maas-controller-577f596567-xwt65" Oct 30 18:48:31.683425 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.683381 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpdsj\" (UniqueName: \"kubernetes.io/projected/01ec44aa-a128-48a4-9d58-0f643487f5ab-kube-api-access-zpdsj\") pod \"maas-controller-577f596567-xwt65\" (UID: \"01ec44aa-a128-48a4-9d58-0f643487f5ab\") " pod="opendatahub/maas-controller-577f596567-xwt65" Oct 30 18:48:31.683425 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.683412 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01ec44aa-a128-48a4-9d58-0f643487f5ab-webhook-cert\") pod \"maas-controller-577f596567-xwt65\" (UID: \"01ec44aa-a128-48a4-9d58-0f643487f5ab\") " pod="opendatahub/maas-controller-577f596567-xwt65" Oct 30 18:48:31.685910 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.685886 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01ec44aa-a128-48a4-9d58-0f643487f5ab-metrics-certs\") pod \"maas-controller-577f596567-xwt65\" (UID: \"01ec44aa-a128-48a4-9d58-0f643487f5ab\") " pod="opendatahub/maas-controller-577f596567-xwt65" Oct 30 18:48:31.686020 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.685953 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01ec44aa-a128-48a4-9d58-0f643487f5ab-webhook-cert\") pod \"maas-controller-577f596567-xwt65\" (UID: \"01ec44aa-a128-48a4-9d58-0f643487f5ab\") " pod="opendatahub/maas-controller-577f596567-xwt65" Oct 30 18:48:31.739712 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.739689 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpdsj\" (UniqueName: \"kubernetes.io/projected/01ec44aa-a128-48a4-9d58-0f643487f5ab-kube-api-access-zpdsj\") pod \"maas-controller-577f596567-xwt65\" (UID: \"01ec44aa-a128-48a4-9d58-0f643487f5ab\") " pod="opendatahub/maas-controller-577f596567-xwt65" Oct 30 18:48:31.764809 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.764776 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-xwt65" Oct 30 18:48:31.765144 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.765114 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:31.950888 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.950808 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:31.984263 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.984235 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-994975fdf-8p5jh"] Oct 30 18:48:31.988754 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:31.988734 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-994975fdf-8p5jh" Oct 30 18:48:32.045455 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:32.045425 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-994975fdf-8p5jh"] Oct 30 18:48:32.087365 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:32.087339 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5264ce31-dc5b-42ed-a76e-631aee315933-webhook-cert\") pod \"maas-controller-994975fdf-8p5jh\" (UID: \"5264ce31-dc5b-42ed-a76e-631aee315933\") " pod="opendatahub/maas-controller-994975fdf-8p5jh" Oct 30 18:48:32.087519 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:32.087382 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5264ce31-dc5b-42ed-a76e-631aee315933-metrics-certs\") pod \"maas-controller-994975fdf-8p5jh\" (UID: \"5264ce31-dc5b-42ed-a76e-631aee315933\") " pod="opendatahub/maas-controller-994975fdf-8p5jh" Oct 30 18:48:32.087519 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:32.087455 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f58pm\" (UniqueName: \"kubernetes.io/projected/5264ce31-dc5b-42ed-a76e-631aee315933-kube-api-access-f58pm\") pod \"maas-controller-994975fdf-8p5jh\" (UID: \"5264ce31-dc5b-42ed-a76e-631aee315933\") " pod="opendatahub/maas-controller-994975fdf-8p5jh" Oct 30 18:48:32.162833 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:32.162721 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-577f596567-xwt65"] Oct 30 18:48:32.165389 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:48:32.165367 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod01ec44aa_a128_48a4_9d58_0f643487f5ab.slice/crio-476e3e7ebd248f80461aa29e177f46b43d1de64a19919496cbed70be11b2d099 WatchSource:0}: Error finding container 476e3e7ebd248f80461aa29e177f46b43d1de64a19919496cbed70be11b2d099: Status 404 returned error can't find the container with id 476e3e7ebd248f80461aa29e177f46b43d1de64a19919496cbed70be11b2d099 Oct 30 18:48:32.188415 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:32.188394 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5264ce31-dc5b-42ed-a76e-631aee315933-metrics-certs\") pod \"maas-controller-994975fdf-8p5jh\" (UID: \"5264ce31-dc5b-42ed-a76e-631aee315933\") " pod="opendatahub/maas-controller-994975fdf-8p5jh" Oct 30 18:48:32.188508 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:32.188422 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f58pm\" (UniqueName: \"kubernetes.io/projected/5264ce31-dc5b-42ed-a76e-631aee315933-kube-api-access-f58pm\") pod \"maas-controller-994975fdf-8p5jh\" (UID: \"5264ce31-dc5b-42ed-a76e-631aee315933\") " pod="opendatahub/maas-controller-994975fdf-8p5jh" Oct 30 18:48:32.188546 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:32.188509 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5264ce31-dc5b-42ed-a76e-631aee315933-webhook-cert\") pod \"maas-controller-994975fdf-8p5jh\" (UID: \"5264ce31-dc5b-42ed-a76e-631aee315933\") " pod="opendatahub/maas-controller-994975fdf-8p5jh" Oct 30 18:48:32.190876 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:32.190856 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5264ce31-dc5b-42ed-a76e-631aee315933-webhook-cert\") pod \"maas-controller-994975fdf-8p5jh\" (UID: \"5264ce31-dc5b-42ed-a76e-631aee315933\") " pod="opendatahub/maas-controller-994975fdf-8p5jh" Oct 30 18:48:32.190966 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:32.190895 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5264ce31-dc5b-42ed-a76e-631aee315933-metrics-certs\") pod \"maas-controller-994975fdf-8p5jh\" (UID: \"5264ce31-dc5b-42ed-a76e-631aee315933\") " pod="opendatahub/maas-controller-994975fdf-8p5jh" Oct 30 18:48:32.245272 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:32.245219 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f58pm\" (UniqueName: \"kubernetes.io/projected/5264ce31-dc5b-42ed-a76e-631aee315933-kube-api-access-f58pm\") pod \"maas-controller-994975fdf-8p5jh\" (UID: \"5264ce31-dc5b-42ed-a76e-631aee315933\") " pod="opendatahub/maas-controller-994975fdf-8p5jh" Oct 30 18:48:32.302322 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:32.302298 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-994975fdf-8p5jh" Oct 30 18:48:32.669477 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:32.669447 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-994975fdf-8p5jh"] Oct 30 18:48:32.672224 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:48:32.672199 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5264ce31_dc5b_42ed_a76e_631aee315933.slice/crio-f2ea90a2e452b42353eba57811bd8ac5a6575e32d5a47a9cd25958aa54a77e88 WatchSource:0}: Error finding container f2ea90a2e452b42353eba57811bd8ac5a6575e32d5a47a9cd25958aa54a77e88: Status 404 returned error can't find the container with id f2ea90a2e452b42353eba57811bd8ac5a6575e32d5a47a9cd25958aa54a77e88 Oct 30 18:48:33.081076 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:33.080992 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-994975fdf-8p5jh" event={"ID":"5264ce31-dc5b-42ed-a76e-631aee315933","Type":"ContainerStarted","Data":"f2ea90a2e452b42353eba57811bd8ac5a6575e32d5a47a9cd25958aa54a77e88"} Oct 30 18:48:33.082078 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:33.082050 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-xwt65" event={"ID":"01ec44aa-a128-48a4-9d58-0f643487f5ab","Type":"ContainerStarted","Data":"476e3e7ebd248f80461aa29e177f46b43d1de64a19919496cbed70be11b2d099"} Oct 30 18:48:37.105171 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:37.105135 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-994975fdf-8p5jh" event={"ID":"5264ce31-dc5b-42ed-a76e-631aee315933","Type":"ContainerStarted","Data":"e71b6ad01758518a08d45087a2a36d510b62c9249007e44425a848c380610233"} Oct 30 18:48:37.105638 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:37.105279 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-994975fdf-8p5jh" Oct 30 18:48:37.106853 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:37.106833 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-xwt65" event={"ID":"01ec44aa-a128-48a4-9d58-0f643487f5ab","Type":"ContainerStarted","Data":"e144898275913f103f0127ec1d816ee523605501d251f4178654cf5be434c92c"} Oct 30 18:48:37.106958 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:37.106947 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-577f596567-xwt65" Oct 30 18:48:37.219528 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:37.219358 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-994975fdf-8p5jh" podStartSLOduration=1.9482987 podStartE2EDuration="6.21934166s" totalImagesPullingTime="4.27104296s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:37.21815609 +0000 UTC m=+587.708283911" watchObservedRunningTime="2026-10-30 18:48:37.21934166 +0000 UTC m=+587.709469481" Oct 30 18:48:37.336822 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:37.336717 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/maas-controller-577f596567-xwt65" podStartSLOduration=1.548319165 podStartE2EDuration="6.336698639s" totalImagesPullingTime="4.788379474s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:37.336596164 +0000 UTC m=+587.826723983" watchObservedRunningTime="2026-10-30 18:48:37.336698639 +0000 UTC m=+587.826826461" Oct 30 18:48:37.370856 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:37.370829 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:37.553916 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:37.553842 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:43.181643 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.181607 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5"] Oct 30 18:48:43.187729 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.186105 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" Oct 30 18:48:43.242356 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.242325 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5"] Oct 30 18:48:43.279436 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.279406 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/3b6fb6f1-df56-4615-b889-13324b291952-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-9n2r5\" (UID: \"3b6fb6f1-df56-4615-b889-13324b291952\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" Oct 30 18:48:43.279631 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.279612 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3b6fb6f1-df56-4615-b889-13324b291952-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-9n2r5\" (UID: \"3b6fb6f1-df56-4615-b889-13324b291952\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" Oct 30 18:48:43.279690 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.279638 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b6fb6f1-df56-4615-b889-13324b291952-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-9n2r5\" (UID: \"3b6fb6f1-df56-4615-b889-13324b291952\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" Oct 30 18:48:43.279690 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.279659 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mdd4f\" (UniqueName: \"kubernetes.io/projected/3b6fb6f1-df56-4615-b889-13324b291952-kube-api-access-mdd4f\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-9n2r5\" (UID: \"3b6fb6f1-df56-4615-b889-13324b291952\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" Oct 30 18:48:43.290332 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.290308 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-ca-bundle\"" Oct 30 18:48:43.290478 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.290310 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-logs-tenancy-proxy-tls\"" Oct 30 18:48:43.290478 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.290301 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-source\"" Oct 30 18:48:43.290478 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.290382 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"usage-tenancy-proxy-dockercfg-bxdrf\"" Oct 30 18:48:43.380212 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.380180 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3b6fb6f1-df56-4615-b889-13324b291952-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-9n2r5\" (UID: \"3b6fb6f1-df56-4615-b889-13324b291952\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" Oct 30 18:48:43.380212 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.380211 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b6fb6f1-df56-4615-b889-13324b291952-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-9n2r5\" (UID: \"3b6fb6f1-df56-4615-b889-13324b291952\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" Oct 30 18:48:43.380442 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.380230 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mdd4f\" (UniqueName: \"kubernetes.io/projected/3b6fb6f1-df56-4615-b889-13324b291952-kube-api-access-mdd4f\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-9n2r5\" (UID: \"3b6fb6f1-df56-4615-b889-13324b291952\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" Oct 30 18:48:43.380442 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.380270 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/3b6fb6f1-df56-4615-b889-13324b291952-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-9n2r5\" (UID: \"3b6fb6f1-df56-4615-b889-13324b291952\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" Oct 30 18:48:43.380442 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:48:43.380358 5894 secret.go:189] Couldn't get secret opendatahub/usage-logs-tenancy-proxy-tls: secret "usage-logs-tenancy-proxy-tls" not found Oct 30 18:48:43.380442 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:48:43.380396 5894 configmap.go:193] Couldn't get configMap opendatahub/usage-tenancy-proxy-source: configmap "usage-tenancy-proxy-source" not found Oct 30 18:48:43.380658 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:48:43.380444 5894 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3b6fb6f1-df56-4615-b889-13324b291952-tls-cert podName:3b6fb6f1-df56-4615-b889-13324b291952 nodeName:}" failed. No retries permitted until 2026-10-30 18:48:43.880421118 +0000 UTC m=+594.370548935 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-cert" (UniqueName: "kubernetes.io/secret/3b6fb6f1-df56-4615-b889-13324b291952-tls-cert") pod "usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" (UID: "3b6fb6f1-df56-4615-b889-13324b291952") : secret "usage-logs-tenancy-proxy-tls" not found Oct 30 18:48:43.380658 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:48:43.380467 5894 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/3b6fb6f1-df56-4615-b889-13324b291952-proxy-source podName:3b6fb6f1-df56-4615-b889-13324b291952 nodeName:}" failed. No retries permitted until 2026-10-30 18:48:43.880453211 +0000 UTC m=+594.370581009 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-source" (UniqueName: "kubernetes.io/configmap/3b6fb6f1-df56-4615-b889-13324b291952-proxy-source") pod "usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" (UID: "3b6fb6f1-df56-4615-b889-13324b291952") : configmap "usage-tenancy-proxy-source" not found Oct 30 18:48:43.380829 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.380809 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3b6fb6f1-df56-4615-b889-13324b291952-service-ca\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-9n2r5\" (UID: \"3b6fb6f1-df56-4615-b889-13324b291952\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" Oct 30 18:48:43.388683 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.388657 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:43.439896 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.439825 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mdd4f\" (UniqueName: \"kubernetes.io/projected/3b6fb6f1-df56-4615-b889-13324b291952-kube-api-access-mdd4f\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-9n2r5\" (UID: \"3b6fb6f1-df56-4615-b889-13324b291952\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" Oct 30 18:48:43.560092 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.560063 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:48:43.884356 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.884252 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/3b6fb6f1-df56-4615-b889-13324b291952-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-9n2r5\" (UID: \"3b6fb6f1-df56-4615-b889-13324b291952\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" Oct 30 18:48:43.884529 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.884358 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3b6fb6f1-df56-4615-b889-13324b291952-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-9n2r5\" (UID: \"3b6fb6f1-df56-4615-b889-13324b291952\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" Oct 30 18:48:43.885079 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.885052 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-source\" (UniqueName: \"kubernetes.io/configmap/3b6fb6f1-df56-4615-b889-13324b291952-proxy-source\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-9n2r5\" (UID: \"3b6fb6f1-df56-4615-b889-13324b291952\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" Oct 30 18:48:43.886675 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:43.886654 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/3b6fb6f1-df56-4615-b889-13324b291952-tls-cert\") pod \"usage-logs-tenancy-proxy-7d94f55f4b-9n2r5\" (UID: \"3b6fb6f1-df56-4615-b889-13324b291952\") " pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" Oct 30 18:48:44.100797 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:44.100764 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" Oct 30 18:48:44.470594 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:44.470396 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5"] Oct 30 18:48:44.473647 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:48:44.473618 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b6fb6f1_df56_4615_b889_13324b291952.slice/crio-c8bf481aeb55c728ebdf085c040630538d381f08dc7c4c7d939bcb4d2aef3087 WatchSource:0}: Error finding container c8bf481aeb55c728ebdf085c040630538d381f08dc7c4c7d939bcb4d2aef3087: Status 404 returned error can't find the container with id c8bf481aeb55c728ebdf085c040630538d381f08dc7c4c7d939bcb4d2aef3087 Oct 30 18:48:45.143393 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:45.143361 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" event={"ID":"3b6fb6f1-df56-4615-b889-13324b291952","Type":"ContainerStarted","Data":"c8bf481aeb55c728ebdf085c040630538d381f08dc7c4c7d939bcb4d2aef3087"} Oct 30 18:48:48.113101 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:48.113067 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-577f596567-xwt65" Oct 30 18:48:48.113549 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:48.113145 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-994975fdf-8p5jh" Oct 30 18:48:48.279004 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:48.278970 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-577f596567-xwt65"] Oct 30 18:48:48.279205 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:48.279162 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-577f596567-xwt65" podUID="01ec44aa-a128-48a4-9d58-0f643487f5ab" containerName="manager" containerID="cri-o://e144898275913f103f0127ec1d816ee523605501d251f4178654cf5be434c92c" gracePeriod=30 Oct 30 18:48:48.518983 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:48.518960 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-xwt65" Oct 30 18:48:48.625916 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:48.625882 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/01ec44aa-a128-48a4-9d58-0f643487f5ab-webhook-cert\" (UniqueName: \"kubernetes.io/secret/01ec44aa-a128-48a4-9d58-0f643487f5ab-webhook-cert\") pod \"01ec44aa-a128-48a4-9d58-0f643487f5ab\" (UID: \"01ec44aa-a128-48a4-9d58-0f643487f5ab\") " Oct 30 18:48:48.626093 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:48.625949 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/01ec44aa-a128-48a4-9d58-0f643487f5ab-metrics-certs\" (UniqueName: \"kubernetes.io/secret/01ec44aa-a128-48a4-9d58-0f643487f5ab-metrics-certs\") pod \"01ec44aa-a128-48a4-9d58-0f643487f5ab\" (UID: \"01ec44aa-a128-48a4-9d58-0f643487f5ab\") " Oct 30 18:48:48.626093 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:48.625993 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/01ec44aa-a128-48a4-9d58-0f643487f5ab-kube-api-access-zpdsj\" (UniqueName: \"kubernetes.io/projected/01ec44aa-a128-48a4-9d58-0f643487f5ab-kube-api-access-zpdsj\") pod \"01ec44aa-a128-48a4-9d58-0f643487f5ab\" (UID: \"01ec44aa-a128-48a4-9d58-0f643487f5ab\") " Oct 30 18:48:48.628521 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:48.628488 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/01ec44aa-a128-48a4-9d58-0f643487f5ab-kube-api-access-zpdsj" pod "01ec44aa-a128-48a4-9d58-0f643487f5ab" (UID: "01ec44aa-a128-48a4-9d58-0f643487f5ab"). InnerVolumeSpecName "kube-api-access-zpdsj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:48:48.628767 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:48.628725 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01ec44aa-a128-48a4-9d58-0f643487f5ab-webhook-cert" pod "01ec44aa-a128-48a4-9d58-0f643487f5ab" (UID: "01ec44aa-a128-48a4-9d58-0f643487f5ab"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:48:48.628767 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:48.628750 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/01ec44aa-a128-48a4-9d58-0f643487f5ab-metrics-certs" pod "01ec44aa-a128-48a4-9d58-0f643487f5ab" (UID: "01ec44aa-a128-48a4-9d58-0f643487f5ab"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:48:48.726959 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:48.726928 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zpdsj\" (UniqueName: \"kubernetes.io/projected/01ec44aa-a128-48a4-9d58-0f643487f5ab-kube-api-access-zpdsj\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:48:48.726959 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:48.726958 5894 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/01ec44aa-a128-48a4-9d58-0f643487f5ab-webhook-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:48:48.726959 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:48.726968 5894 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/01ec44aa-a128-48a4-9d58-0f643487f5ab-metrics-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:48:49.166497 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:49.166458 5894 generic.go:356] "Generic (PLEG): container finished" podID="01ec44aa-a128-48a4-9d58-0f643487f5ab" containerID="e144898275913f103f0127ec1d816ee523605501d251f4178654cf5be434c92c" exitCode=0 Oct 30 18:48:49.166967 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:49.166517 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-xwt65" event={"ID":"01ec44aa-a128-48a4-9d58-0f643487f5ab","Type":"ContainerDied","Data":"e144898275913f103f0127ec1d816ee523605501d251f4178654cf5be434c92c"} Oct 30 18:48:49.166967 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:49.166569 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-577f596567-xwt65" event={"ID":"01ec44aa-a128-48a4-9d58-0f643487f5ab","Type":"ContainerDied","Data":"476e3e7ebd248f80461aa29e177f46b43d1de64a19919496cbed70be11b2d099"} Oct 30 18:48:49.166967 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:49.166626 5894 scope.go:122] "RemoveContainer" containerID="e144898275913f103f0127ec1d816ee523605501d251f4178654cf5be434c92c" Oct 30 18:48:49.166967 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:49.166786 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-577f596567-xwt65" Oct 30 18:48:49.184538 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:49.184430 5894 scope.go:122] "RemoveContainer" containerID="e144898275913f103f0127ec1d816ee523605501d251f4178654cf5be434c92c" Oct 30 18:48:49.184831 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:48:49.184742 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e144898275913f103f0127ec1d816ee523605501d251f4178654cf5be434c92c\": container with ID starting with e144898275913f103f0127ec1d816ee523605501d251f4178654cf5be434c92c not found: ID does not exist" containerID="e144898275913f103f0127ec1d816ee523605501d251f4178654cf5be434c92c" Oct 30 18:48:49.184831 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:49.184782 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e144898275913f103f0127ec1d816ee523605501d251f4178654cf5be434c92c"} err="failed to get container status \"e144898275913f103f0127ec1d816ee523605501d251f4178654cf5be434c92c\": rpc error: code = NotFound desc = could not find container \"e144898275913f103f0127ec1d816ee523605501d251f4178654cf5be434c92c\": container with ID starting with e144898275913f103f0127ec1d816ee523605501d251f4178654cf5be434c92c not found: ID does not exist" Oct 30 18:48:49.335547 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:49.335499 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-577f596567-xwt65"] Oct 30 18:48:49.383062 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:49.383029 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-577f596567-xwt65"] Oct 30 18:48:49.650771 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:49.650740 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="01ec44aa-a128-48a4-9d58-0f643487f5ab" path="/var/lib/kubelet/pods/01ec44aa-a128-48a4-9d58-0f643487f5ab/volumes" Oct 30 18:48:49.712493 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:49.712465 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 18:48:49.714872 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:49.714853 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 18:48:50.123893 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:48:50.123785 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:48:58.216601 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:58.216497 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" event={"ID":"3b6fb6f1-df56-4615-b889-13324b291952","Type":"ContainerStarted","Data":"c73d6c286842ff64bc85fe145e7a1fa402fea3fa92b17038a90ce2bd870c6b60"} Oct 30 18:48:58.217002 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:58.216623 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" Oct 30 18:48:58.331098 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:48:58.331037 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" podStartSLOduration=1.881075203 podStartE2EDuration="15.331021634s" totalImagesPullingTime="13.449946431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:48:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:48:58.330276208 +0000 UTC m=+608.820404029" watchObservedRunningTime="2026-10-30 18:48:58.331021634 +0000 UTC m=+608.821149454" Oct 30 18:49:06.500039 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:06.499988 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:49:06.691955 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:06.691900 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:49:08.537598 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:08.537532 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:49:08.712975 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:08.712947 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:49:08.883287 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:08.883212 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:49:09.226828 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.226794 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/usage-logs-tenancy-proxy-7d94f55f4b-9n2r5" Oct 30 18:49:09.605895 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.604128 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-msjhg"] Oct 30 18:49:09.605895 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.605047 5894 cpu_manager.go:414] "containerMap: removing container" podUID="01ec44aa-a128-48a4-9d58-0f643487f5ab" containerName="manager" Oct 30 18:49:09.605895 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.605067 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="01ec44aa-a128-48a4-9d58-0f643487f5ab" containerName="manager" Oct 30 18:49:09.605895 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.605242 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="01ec44aa-a128-48a4-9d58-0f643487f5ab" containerName="manager" Oct 30 18:49:09.610512 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.610483 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" Oct 30 18:49:09.663359 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.663330 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-msjhg"] Oct 30 18:49:09.711734 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.711705 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Oct 30 18:49:09.711881 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.711705 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Oct 30 18:49:09.711881 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.711825 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-9wflp\"" Oct 30 18:49:09.717095 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.717075 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49d04e0a-032d-4491-a4ea-be136c886f8a-metrics-certs\") pod \"maas-api-5c48877c75-msjhg\" (UID: \"49d04e0a-032d-4491-a4ea-be136c886f8a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" Oct 30 18:49:09.717186 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.717103 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d04e0a-032d-4491-a4ea-be136c886f8a-maas-api-tls\") pod \"maas-api-5c48877c75-msjhg\" (UID: \"49d04e0a-032d-4491-a4ea-be136c886f8a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" Oct 30 18:49:09.717223 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.717198 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rnvx5\" (UniqueName: \"kubernetes.io/projected/49d04e0a-032d-4491-a4ea-be136c886f8a-kube-api-access-rnvx5\") pod \"maas-api-5c48877c75-msjhg\" (UID: \"49d04e0a-032d-4491-a4ea-be136c886f8a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" Oct 30 18:49:09.717256 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.717232 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d04e0a-032d-4491-a4ea-be136c886f8a-openshift-service-ca\") pod \"maas-api-5c48877c75-msjhg\" (UID: \"49d04e0a-032d-4491-a4ea-be136c886f8a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" Oct 30 18:49:09.818552 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.818515 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49d04e0a-032d-4491-a4ea-be136c886f8a-metrics-certs\") pod \"maas-api-5c48877c75-msjhg\" (UID: \"49d04e0a-032d-4491-a4ea-be136c886f8a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" Oct 30 18:49:09.818552 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.818553 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d04e0a-032d-4491-a4ea-be136c886f8a-maas-api-tls\") pod \"maas-api-5c48877c75-msjhg\" (UID: \"49d04e0a-032d-4491-a4ea-be136c886f8a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" Oct 30 18:49:09.818809 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.818615 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rnvx5\" (UniqueName: \"kubernetes.io/projected/49d04e0a-032d-4491-a4ea-be136c886f8a-kube-api-access-rnvx5\") pod \"maas-api-5c48877c75-msjhg\" (UID: \"49d04e0a-032d-4491-a4ea-be136c886f8a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" Oct 30 18:49:09.818809 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.818639 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d04e0a-032d-4491-a4ea-be136c886f8a-openshift-service-ca\") pod \"maas-api-5c48877c75-msjhg\" (UID: \"49d04e0a-032d-4491-a4ea-be136c886f8a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" Oct 30 18:49:09.819296 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.819272 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d04e0a-032d-4491-a4ea-be136c886f8a-openshift-service-ca\") pod \"maas-api-5c48877c75-msjhg\" (UID: \"49d04e0a-032d-4491-a4ea-be136c886f8a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" Oct 30 18:49:09.821103 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.821082 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d04e0a-032d-4491-a4ea-be136c886f8a-maas-api-tls\") pod \"maas-api-5c48877c75-msjhg\" (UID: \"49d04e0a-032d-4491-a4ea-be136c886f8a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" Oct 30 18:49:09.821171 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.821082 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49d04e0a-032d-4491-a4ea-be136c886f8a-metrics-certs\") pod \"maas-api-5c48877c75-msjhg\" (UID: \"49d04e0a-032d-4491-a4ea-be136c886f8a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" Oct 30 18:49:09.875621 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.875544 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rnvx5\" (UniqueName: \"kubernetes.io/projected/49d04e0a-032d-4491-a4ea-be136c886f8a-kube-api-access-rnvx5\") pod \"maas-api-5c48877c75-msjhg\" (UID: \"49d04e0a-032d-4491-a4ea-be136c886f8a\") " pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" Oct 30 18:49:09.921210 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:09.921179 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" Oct 30 18:49:10.290810 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:10.290780 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-msjhg"] Oct 30 18:49:10.293694 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:49:10.293667 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49d04e0a_032d_4491_a4ea_be136c886f8a.slice/crio-badd9d1904f714eb5732058c0bca0736921024100b9ba9123eafa10778c2bd91 WatchSource:0}: Error finding container badd9d1904f714eb5732058c0bca0736921024100b9ba9123eafa10778c2bd91: Status 404 returned error can't find the container with id badd9d1904f714eb5732058c0bca0736921024100b9ba9123eafa10778c2bd91 Oct 30 18:49:11.275427 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:11.275390 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" event={"ID":"49d04e0a-032d-4491-a4ea-be136c886f8a","Type":"ContainerStarted","Data":"badd9d1904f714eb5732058c0bca0736921024100b9ba9123eafa10778c2bd91"} Oct 30 18:49:15.089187 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:15.089152 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:49:15.295943 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:15.295909 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" event={"ID":"49d04e0a-032d-4491-a4ea-be136c886f8a","Type":"ContainerStarted","Data":"b9dfea8501499f8cba4e17479118518ba4c11a0c884b00e6ebf6b4b78d1ca1dd"} Oct 30 18:49:15.296100 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:15.295970 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" Oct 30 18:49:15.413023 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:15.412976 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" podStartSLOduration=2.33164246 podStartE2EDuration="6.412962119s" totalImagesPullingTime="4.081319659s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:49:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:49:15.412165219 +0000 UTC m=+625.902293039" watchObservedRunningTime="2026-10-30 18:49:15.412962119 +0000 UTC m=+625.903089938" Oct 30 18:49:20.116302 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:49:20.116260 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:49:21.123152 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:21.123095 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:49:21.295639 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:21.295606 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:49:21.305539 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:21.305517 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" Oct 30 18:49:22.498758 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:22.498725 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-7qw75"] Oct 30 18:49:22.502337 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:22.502320 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-7qw75" Oct 30 18:49:22.558956 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:22.558928 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-7qw75"] Oct 30 18:49:22.629675 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:22.629645 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43305b9a-736e-472f-ae60-9001cfc5f237-cert\") pod \"llmisvc-controller-manager-6466c554d8-7qw75\" (UID: \"43305b9a-736e-472f-ae60-9001cfc5f237\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-7qw75" Oct 30 18:49:22.629842 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:22.629710 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pd6bs\" (UniqueName: \"kubernetes.io/projected/43305b9a-736e-472f-ae60-9001cfc5f237-kube-api-access-pd6bs\") pod \"llmisvc-controller-manager-6466c554d8-7qw75\" (UID: \"43305b9a-736e-472f-ae60-9001cfc5f237\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-7qw75" Oct 30 18:49:22.730745 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:22.730709 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pd6bs\" (UniqueName: \"kubernetes.io/projected/43305b9a-736e-472f-ae60-9001cfc5f237-kube-api-access-pd6bs\") pod \"llmisvc-controller-manager-6466c554d8-7qw75\" (UID: \"43305b9a-736e-472f-ae60-9001cfc5f237\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-7qw75" Oct 30 18:49:22.730896 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:22.730768 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43305b9a-736e-472f-ae60-9001cfc5f237-cert\") pod \"llmisvc-controller-manager-6466c554d8-7qw75\" (UID: \"43305b9a-736e-472f-ae60-9001cfc5f237\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-7qw75" Oct 30 18:49:22.733295 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:22.733271 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/43305b9a-736e-472f-ae60-9001cfc5f237-cert\") pod \"llmisvc-controller-manager-6466c554d8-7qw75\" (UID: \"43305b9a-736e-472f-ae60-9001cfc5f237\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-7qw75" Oct 30 18:49:22.788116 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:22.788022 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pd6bs\" (UniqueName: \"kubernetes.io/projected/43305b9a-736e-472f-ae60-9001cfc5f237-kube-api-access-pd6bs\") pod \"llmisvc-controller-manager-6466c554d8-7qw75\" (UID: \"43305b9a-736e-472f-ae60-9001cfc5f237\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-7qw75" Oct 30 18:49:22.814756 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:22.814731 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-7qw75" Oct 30 18:49:23.185536 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:23.185505 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-7qw75"] Oct 30 18:49:23.188759 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:49:23.188730 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43305b9a_736e_472f_ae60_9001cfc5f237.slice/crio-6ef0c71a8d1f018ce4c7fafd302acfb82cc3f63d5181fa7a47af7486c1ca7bf6 WatchSource:0}: Error finding container 6ef0c71a8d1f018ce4c7fafd302acfb82cc3f63d5181fa7a47af7486c1ca7bf6: Status 404 returned error can't find the container with id 6ef0c71a8d1f018ce4c7fafd302acfb82cc3f63d5181fa7a47af7486c1ca7bf6 Oct 30 18:49:23.332832 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:23.332800 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-7qw75" event={"ID":"43305b9a-736e-472f-ae60-9001cfc5f237","Type":"ContainerStarted","Data":"6ef0c71a8d1f018ce4c7fafd302acfb82cc3f63d5181fa7a47af7486c1ca7bf6"} Oct 30 18:49:24.341496 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:24.341427 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-7qw75" event={"ID":"43305b9a-736e-472f-ae60-9001cfc5f237","Type":"ContainerStarted","Data":"570cfc33da444222cffc983efd110c7b7f1b40a95593af1609dd1799eaa1aad0"} Oct 30 18:49:24.341904 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:24.341555 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-7qw75" Oct 30 18:49:24.455136 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:24.455087 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-7qw75" podStartSLOduration=1.427893514 podStartE2EDuration="2.455072751s" totalImagesPullingTime="1.027179237s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:49:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:49:24.454996515 +0000 UTC m=+634.945124336" watchObservedRunningTime="2026-10-30 18:49:24.455072751 +0000 UTC m=+634.945200570" Oct 30 18:49:29.129466 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:29.129400 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:49:50.116812 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:49:50.116735 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:49:55.348694 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:55.348663 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-7qw75" Oct 30 18:49:55.513003 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:55.512968 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256"] Oct 30 18:49:55.513234 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:55.513195 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256" podUID="ee74582d-1edf-412c-9a76-459d08a10cc3" containerName="manager" containerID="cri-o://3936573f2d464ae43e0cbe7e95c6a198a42054f70afdadc04efd888cb206eccf" gracePeriod=30 Oct 30 18:49:55.766167 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:55.766141 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256" Oct 30 18:49:55.898493 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:55.898461 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee74582d-1edf-412c-9a76-459d08a10cc3-cert\" (UniqueName: \"kubernetes.io/secret/ee74582d-1edf-412c-9a76-459d08a10cc3-cert\") pod \"ee74582d-1edf-412c-9a76-459d08a10cc3\" (UID: \"ee74582d-1edf-412c-9a76-459d08a10cc3\") " Oct 30 18:49:55.898662 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:55.898634 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee74582d-1edf-412c-9a76-459d08a10cc3-kube-api-access-jt5zq\" (UniqueName: \"kubernetes.io/projected/ee74582d-1edf-412c-9a76-459d08a10cc3-kube-api-access-jt5zq\") pod \"ee74582d-1edf-412c-9a76-459d08a10cc3\" (UID: \"ee74582d-1edf-412c-9a76-459d08a10cc3\") " Oct 30 18:49:55.900587 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:55.900548 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee74582d-1edf-412c-9a76-459d08a10cc3-cert" pod "ee74582d-1edf-412c-9a76-459d08a10cc3" (UID: "ee74582d-1edf-412c-9a76-459d08a10cc3"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 18:49:55.900756 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:55.900700 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee74582d-1edf-412c-9a76-459d08a10cc3-kube-api-access-jt5zq" pod "ee74582d-1edf-412c-9a76-459d08a10cc3" (UID: "ee74582d-1edf-412c-9a76-459d08a10cc3"). InnerVolumeSpecName "kube-api-access-jt5zq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:49:55.999415 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:55.999380 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt5zq\" (UniqueName: \"kubernetes.io/projected/ee74582d-1edf-412c-9a76-459d08a10cc3-kube-api-access-jt5zq\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:49:55.999415 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:55.999409 5894 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ee74582d-1edf-412c-9a76-459d08a10cc3-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:49:56.389368 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:56.389269 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:49:56.483085 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:56.483052 5894 generic.go:356] "Generic (PLEG): container finished" podID="ee74582d-1edf-412c-9a76-459d08a10cc3" containerID="3936573f2d464ae43e0cbe7e95c6a198a42054f70afdadc04efd888cb206eccf" exitCode=0 Oct 30 18:49:56.483248 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:56.483101 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256" Oct 30 18:49:56.483248 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:56.483137 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256" event={"ID":"ee74582d-1edf-412c-9a76-459d08a10cc3","Type":"ContainerDied","Data":"3936573f2d464ae43e0cbe7e95c6a198a42054f70afdadc04efd888cb206eccf"} Oct 30 18:49:56.483248 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:56.483190 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256" event={"ID":"ee74582d-1edf-412c-9a76-459d08a10cc3","Type":"ContainerDied","Data":"4c96a83f00ec403b066e510f05196323b7ea85e89fa85632eb5efc73962d7244"} Oct 30 18:49:56.483248 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:56.483211 5894 scope.go:122] "RemoveContainer" containerID="3936573f2d464ae43e0cbe7e95c6a198a42054f70afdadc04efd888cb206eccf" Oct 30 18:49:56.492631 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:56.492612 5894 scope.go:122] "RemoveContainer" containerID="3936573f2d464ae43e0cbe7e95c6a198a42054f70afdadc04efd888cb206eccf" Oct 30 18:49:56.492910 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:49:56.492891 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3936573f2d464ae43e0cbe7e95c6a198a42054f70afdadc04efd888cb206eccf\": container with ID starting with 3936573f2d464ae43e0cbe7e95c6a198a42054f70afdadc04efd888cb206eccf not found: ID does not exist" containerID="3936573f2d464ae43e0cbe7e95c6a198a42054f70afdadc04efd888cb206eccf" Oct 30 18:49:56.492984 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:56.492918 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3936573f2d464ae43e0cbe7e95c6a198a42054f70afdadc04efd888cb206eccf"} err="failed to get container status \"3936573f2d464ae43e0cbe7e95c6a198a42054f70afdadc04efd888cb206eccf\": rpc error: code = NotFound desc = could not find container \"3936573f2d464ae43e0cbe7e95c6a198a42054f70afdadc04efd888cb206eccf\": container with ID starting with 3936573f2d464ae43e0cbe7e95c6a198a42054f70afdadc04efd888cb206eccf not found: ID does not exist" Oct 30 18:49:56.586954 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:56.586926 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:49:56.644934 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:56.644906 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256"] Oct 30 18:49:56.694431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:56.694398 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-ql256"] Oct 30 18:49:57.648759 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:49:57.648727 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee74582d-1edf-412c-9a76-459d08a10cc3" path="/var/lib/kubelet/pods/ee74582d-1edf-412c-9a76-459d08a10cc3/volumes" Oct 30 18:50:05.061164 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:05.061107 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:50:13.577917 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:13.577832 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:50:15.341141 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:15.341105 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:50:15.517805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:15.517745 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:50:19.221089 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:19.221055 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:50:20.116432 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:50:20.116393 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:50:22.865038 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:22.865002 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:50:26.522936 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:26.522891 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:50:33.833988 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:33.833926 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:50:34.144003 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:34.143958 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:50:37.310855 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:37.310799 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:50:40.842051 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.842016 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6"] Oct 30 18:50:40.842451 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.842406 5894 cpu_manager.go:414] "containerMap: removing container" podUID="ee74582d-1edf-412c-9a76-459d08a10cc3" containerName="manager" Oct 30 18:50:40.842451 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.842426 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee74582d-1edf-412c-9a76-459d08a10cc3" containerName="manager" Oct 30 18:50:40.842528 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.842519 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee74582d-1edf-412c-9a76-459d08a10cc3" containerName="manager" Oct 30 18:50:40.845705 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.845685 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.897173 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.897136 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.897336 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.897187 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.897336 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.897319 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.897483 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.897366 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.897483 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.897419 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.897609 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.897565 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-qwc8z\"" Oct 30 18:50:40.897671 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.897603 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.897671 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.897641 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.897756 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.897728 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk8s8\" (UniqueName: \"kubernetes.io/projected/187e78e5-00ef-4ee9-97a7-4b6fa262c551-kube-api-access-mk8s8\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.897809 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.897774 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.948887 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.948860 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6"] Oct 30 18:50:40.998736 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.998704 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.998917 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.998755 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.998917 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.998777 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.998917 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.998845 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.998917 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.998869 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.998917 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.998907 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mk8s8\" (UniqueName: \"kubernetes.io/projected/187e78e5-00ef-4ee9-97a7-4b6fa262c551-kube-api-access-mk8s8\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.999174 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.998937 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.999174 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.998981 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.999174 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.999004 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.999327 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.999190 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.999327 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.999222 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.999477 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.999457 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.999563 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.999539 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:40.999838 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:40.999814 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:41.001280 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:41.001257 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:41.001504 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:41.001484 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:41.055243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:41.055208 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:41.055382 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:41.055300 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk8s8\" (UniqueName: \"kubernetes.io/projected/187e78e5-00ef-4ee9-97a7-4b6fa262c551-kube-api-access-mk8s8\") pod \"maas-default-gateway-openshift-default-6db586dd6c-mvch6\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:41.158801 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:41.158769 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:41.580914 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:41.580887 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6"] Oct 30 18:50:41.583615 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:50:41.583559 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod187e78e5_00ef_4ee9_97a7_4b6fa262c551.slice/crio-5650964eaffb1dd910c9ba2b19c0836f20c04390728caf866ff5e3155eba2ad3 WatchSource:0}: Error finding container 5650964eaffb1dd910c9ba2b19c0836f20c04390728caf866ff5e3155eba2ad3: Status 404 returned error can't find the container with id 5650964eaffb1dd910c9ba2b19c0836f20c04390728caf866ff5e3155eba2ad3 Oct 30 18:50:41.585586 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:41.585553 5894 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 18:50:41.696150 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:41.696116 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" event={"ID":"187e78e5-00ef-4ee9-97a7-4b6fa262c551","Type":"ContainerStarted","Data":"5650964eaffb1dd910c9ba2b19c0836f20c04390728caf866ff5e3155eba2ad3"} Oct 30 18:50:42.441403 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:42.441363 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:50:47.566925 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:47.566888 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:50:47.567320 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:47.566976 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:50:47.567320 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:47.567023 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:50:47.727398 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:47.727359 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" event={"ID":"187e78e5-00ef-4ee9-97a7-4b6fa262c551","Type":"ContainerStarted","Data":"95c9810d8b818ec4c75c785f1de172c159729d0f54b735a79bf24390065e32e5"} Oct 30 18:50:47.851279 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:47.851169 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" podStartSLOduration=1.87085266 podStartE2EDuration="7.851150478s" totalImagesPullingTime="5.980297818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:50:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:50:47.849213999 +0000 UTC m=+718.339341825" watchObservedRunningTime="2026-10-30 18:50:47.851150478 +0000 UTC m=+718.341278304" Oct 30 18:50:48.159394 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:48.159354 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:48.160939 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:48.160909 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-mvch6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Oct 30 18:50:48.161023 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:48.160982 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" podUID="187e78e5-00ef-4ee9-97a7-4b6fa262c551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Oct 30 18:50:49.160121 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:49.160083 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-mvch6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Oct 30 18:50:49.160517 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:49.160145 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" podUID="187e78e5-00ef-4ee9-97a7-4b6fa262c551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Oct 30 18:50:50.117277 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:50:50.117190 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:50:50.159844 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:50.159806 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-mvch6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Oct 30 18:50:50.160005 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:50.159865 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" podUID="187e78e5-00ef-4ee9-97a7-4b6fa262c551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Oct 30 18:50:51.159809 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:51.159772 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-mvch6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Oct 30 18:50:51.160213 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:51.159841 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" podUID="187e78e5-00ef-4ee9-97a7-4b6fa262c551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Oct 30 18:50:52.159182 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:52.159140 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-mvch6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Oct 30 18:50:52.159391 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:52.159214 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" podUID="187e78e5-00ef-4ee9-97a7-4b6fa262c551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Oct 30 18:50:53.159508 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:53.159471 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-mvch6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Oct 30 18:50:53.159930 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:53.159537 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" podUID="187e78e5-00ef-4ee9-97a7-4b6fa262c551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Oct 30 18:50:54.159672 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:54.159634 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-mvch6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Oct 30 18:50:54.160156 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:54.159700 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" podUID="187e78e5-00ef-4ee9-97a7-4b6fa262c551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Oct 30 18:50:55.797240 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:55.797208 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:55.797667 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:55.797567 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:55.798382 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:55.798353 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:50:57.384511 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:57.384479 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:50:57.731953 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:57.731924 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:50:58.380521 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:50:58.380466 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:51:08.390967 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:08.390894 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:51:08.573145 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:08.573108 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:51:10.835339 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:10.835293 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr"] Oct 30 18:51:10.842370 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:10.842348 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:10.945736 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:10.945708 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr"] Oct 30 18:51:10.979884 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:10.979847 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b18383d6-7b02-4191-80de-440dc8bbee34-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:10.980040 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:10.979917 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:10.980040 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:10.980007 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18383d6-7b02-4191-80de-440dc8bbee34-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:10.980133 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:10.980047 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:10.980173 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:10.980142 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:10.980210 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:10.980188 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:10.980250 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:10.980206 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18383d6-7b02-4191-80de-440dc8bbee34-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:10.980284 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:10.980262 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:10.980331 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:10.980294 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkgl6\" (UniqueName: \"kubernetes.io/projected/b18383d6-7b02-4191-80de-440dc8bbee34-kube-api-access-bkgl6\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.080876 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.080838 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.081068 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.080905 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.081068 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.080943 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.081068 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.080965 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18383d6-7b02-4191-80de-440dc8bbee34-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.081068 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.080994 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.081068 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.081024 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bkgl6\" (UniqueName: \"kubernetes.io/projected/b18383d6-7b02-4191-80de-440dc8bbee34-kube-api-access-bkgl6\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.081068 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.081052 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b18383d6-7b02-4191-80de-440dc8bbee34-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.081378 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.081132 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.081378 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.081343 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.081488 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.081377 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18383d6-7b02-4191-80de-440dc8bbee34-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.081488 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.081408 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.081628 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.081376 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.081628 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.081612 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.081795 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.081767 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18383d6-7b02-4191-80de-440dc8bbee34-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.083332 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.083311 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.083737 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.083716 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18383d6-7b02-4191-80de-440dc8bbee34-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.137598 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.137510 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkgl6\" (UniqueName: \"kubernetes.io/projected/b18383d6-7b02-4191-80de-440dc8bbee34-kube-api-access-bkgl6\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.137598 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.137526 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b18383d6-7b02-4191-80de-440dc8bbee34-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2lmdr\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.157412 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.157391 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:11.581498 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.581474 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr"] Oct 30 18:51:11.583985 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:51:11.583960 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb18383d6_7b02_4191_80de_440dc8bbee34.slice/crio-2f97e0ed97f0c9e94acddafae4246005ab483766e10d5af5db1aeff6c0050c80 WatchSource:0}: Error finding container 2f97e0ed97f0c9e94acddafae4246005ab483766e10d5af5db1aeff6c0050c80: Status 404 returned error can't find the container with id 2f97e0ed97f0c9e94acddafae4246005ab483766e10d5af5db1aeff6c0050c80 Oct 30 18:51:11.585879 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.585849 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:51:11.585963 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.585908 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:51:11.585963 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.585958 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:51:11.863615 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.863517 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" event={"ID":"b18383d6-7b02-4191-80de-440dc8bbee34","Type":"ContainerStarted","Data":"f260126e58c6d54ff2019510ad08236ff6f17c4fccdd09cbc87e627e3ccb0d4d"} Oct 30 18:51:11.863615 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.863604 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" event={"ID":"b18383d6-7b02-4191-80de-440dc8bbee34","Type":"ContainerStarted","Data":"2f97e0ed97f0c9e94acddafae4246005ab483766e10d5af5db1aeff6c0050c80"} Oct 30 18:51:11.982652 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:11.982565 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" podStartSLOduration=1.982544486 podStartE2EDuration="1.982544486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:51:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:51:11.979130389 +0000 UTC m=+742.469258212" watchObservedRunningTime="2026-10-30 18:51:11.982544486 +0000 UTC m=+742.472672308" Oct 30 18:51:12.158372 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:12.158321 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:12.159923 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:12.159894 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-2lmdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Oct 30 18:51:12.160054 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:12.159960 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" podUID="b18383d6-7b02-4191-80de-440dc8bbee34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Oct 30 18:51:13.158821 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:13.158782 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-2lmdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Oct 30 18:51:13.159209 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:13.158863 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" podUID="b18383d6-7b02-4191-80de-440dc8bbee34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Oct 30 18:51:14.158487 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:14.158454 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-2lmdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Oct 30 18:51:14.158685 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:14.158516 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" podUID="b18383d6-7b02-4191-80de-440dc8bbee34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Oct 30 18:51:15.158764 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:15.158723 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-2lmdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Oct 30 18:51:15.159163 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:15.158794 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" podUID="b18383d6-7b02-4191-80de-440dc8bbee34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Oct 30 18:51:15.758775 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:15.758740 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:51:16.158139 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:16.158106 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-2lmdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Oct 30 18:51:16.158303 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:16.158163 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" podUID="b18383d6-7b02-4191-80de-440dc8bbee34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Oct 30 18:51:17.158668 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:17.158632 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-2lmdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Oct 30 18:51:17.159156 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:17.158693 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" podUID="b18383d6-7b02-4191-80de-440dc8bbee34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Oct 30 18:51:18.158617 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:18.158550 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-2lmdr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body= Oct 30 18:51:18.158793 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:18.158642 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" podUID="b18383d6-7b02-4191-80de-440dc8bbee34" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" Oct 30 18:51:19.906943 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:19.906890 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:19.907776 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:19.907750 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:19.908531 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:19.908513 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:51:20.117152 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:51:20.117112 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:51:30.638014 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:30.637984 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:51:31.146868 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:31.146831 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:51:32.776331 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:32.776299 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:51:32.956954 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:32.956923 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:51:34.310321 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:34.310285 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:51:34.499557 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:34.499523 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:51:50.117217 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:51:50.117183 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:51:58.899943 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:58.899911 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:51:59.113969 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:59.113936 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:51:59.553419 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:59.553382 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:51:59.773611 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:59.772931 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:51:59.952366 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:51:59.952329 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:00.137419 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:00.137367 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:00.321425 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:00.321344 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:00.497214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:00.497181 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:00.678256 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:00.678221 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:00.857625 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:00.857564 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:01.048566 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:01.048479 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:01.234962 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:01.234924 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:01.417036 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:01.417000 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:01.595763 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:01.595728 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:01.773047 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:01.772962 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:03.971155 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:03.971120 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:04.251698 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:04.251602 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:04.437398 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:04.437353 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:04.913535 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:04.913501 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:05.104220 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:05.104181 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:05.297976 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:05.297894 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:05.585606 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:05.585498 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:05.786707 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:05.786680 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:06.083959 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:06.083926 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:06.287025 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:06.286993 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:07.643709 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:07.643680 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:07.845220 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:07.845187 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:08.149668 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:08.149639 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:08.347007 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:08.346968 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:08.547591 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:08.546162 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:09.647113 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:09.647074 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:09.838923 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:09.838891 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5"] Oct 30 18:52:09.842280 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:09.842263 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" Oct 30 18:52:09.861518 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:09.861494 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:09.898538 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:09.898479 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w6-41db7c-serving-cert\"" Oct 30 18:52:09.945044 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:09.945014 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5"] Oct 30 18:52:10.027515 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:10.027484 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lw7c\" (UniqueName: \"kubernetes.io/projected/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-kube-api-access-9lw7c\") pod \"maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5\" (UID: \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" Oct 30 18:52:10.027715 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:10.027522 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-maas-api-tls\") pod \"maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5\" (UID: \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" Oct 30 18:52:10.027715 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:10.027627 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5\" (UID: \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" Oct 30 18:52:10.027825 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:10.027721 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-metrics-certs\") pod \"maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5\" (UID: \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" Oct 30 18:52:10.124278 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:10.124240 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:10.128270 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:10.128244 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5\" (UID: \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" Oct 30 18:52:10.128270 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:10.128276 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-metrics-certs\") pod \"maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5\" (UID: \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" Oct 30 18:52:10.128488 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:10.128335 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lw7c\" (UniqueName: \"kubernetes.io/projected/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-kube-api-access-9lw7c\") pod \"maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5\" (UID: \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" Oct 30 18:52:10.128488 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:10.128372 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-maas-api-tls\") pod \"maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5\" (UID: \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" Oct 30 18:52:10.128488 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:52:10.128464 5894 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-serving-cert: secret "maas-api-e2e-worker-w6-41db7c-serving-cert" not found Oct 30 18:52:10.128634 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:52:10.128532 5894 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-maas-api-tls podName:df3270ce-707c-45f4-a0d9-3dfd137fbdfc nodeName:}" failed. No retries permitted until 2026-10-30 18:52:10.628510202 +0000 UTC m=+801.118638006 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-maas-api-tls") pod "maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" (UID: "df3270ce-707c-45f4-a0d9-3dfd137fbdfc") : secret "maas-api-e2e-worker-w6-41db7c-serving-cert" not found Oct 30 18:52:10.129010 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:10.128987 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-openshift-service-ca\") pod \"maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5\" (UID: \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" Oct 30 18:52:10.130858 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:10.130836 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-metrics-certs\") pod \"maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5\" (UID: \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" Oct 30 18:52:10.186789 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:10.186722 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lw7c\" (UniqueName: \"kubernetes.io/projected/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-kube-api-access-9lw7c\") pod \"maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5\" (UID: \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" Oct 30 18:52:10.320887 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:10.320846 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:10.490781 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:10.490693 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:10.632741 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:10.632709 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-maas-api-tls\") pod \"maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5\" (UID: \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" Oct 30 18:52:10.635190 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:10.635161 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-maas-api-tls\") pod \"maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5\" (UID: \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" Oct 30 18:52:10.756016 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:10.755946 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" Oct 30 18:52:11.127321 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:11.127291 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5"] Oct 30 18:52:11.133199 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:52:11.133156 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf3270ce_707c_45f4_a0d9_3dfd137fbdfc.slice/crio-13f023457d67d734dbf02aadad509ef49f468d074e2dbb7e47b865a9e6f397b9 WatchSource:0}: Error finding container 13f023457d67d734dbf02aadad509ef49f468d074e2dbb7e47b865a9e6f397b9: Status 404 returned error can't find the container with id 13f023457d67d734dbf02aadad509ef49f468d074e2dbb7e47b865a9e6f397b9 Oct 30 18:52:11.162958 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:11.162927 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" event={"ID":"df3270ce-707c-45f4-a0d9-3dfd137fbdfc","Type":"ContainerStarted","Data":"13f023457d67d734dbf02aadad509ef49f468d074e2dbb7e47b865a9e6f397b9"} Oct 30 18:52:12.168900 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:12.168868 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" event={"ID":"df3270ce-707c-45f4-a0d9-3dfd137fbdfc","Type":"ContainerStarted","Data":"71c5374104491265deb4936ab6d2f2cc64b8faddcb989be8c71f4df8ab830edc"} Oct 30 18:52:12.169284 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:12.168938 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" Oct 30 18:52:12.289243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:12.289133 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" podStartSLOduration=2.43375417 podStartE2EDuration="3.289117931s" totalImagesPullingTime="855.363761ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:12.287437993 +0000 UTC m=+802.777565814" watchObservedRunningTime="2026-10-30 18:52:12.289117931 +0000 UTC m=+802.779245750" Oct 30 18:52:13.151083 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:13.151050 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:13.337852 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:13.337825 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:15.052047 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.052015 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699"] Oct 30 18:52:15.055552 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.055535 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" Oct 30 18:52:15.067344 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.067323 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-maas-api-tls\") pod \"maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699\" (UID: \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" Oct 30 18:52:15.067450 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.067356 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699\" (UID: \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" Oct 30 18:52:15.067450 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.067381 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-metrics-certs\") pod \"maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699\" (UID: \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" Oct 30 18:52:15.067552 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.067508 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mcgtf\" (UniqueName: \"kubernetes.io/projected/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-kube-api-access-mcgtf\") pod \"maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699\" (UID: \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" Oct 30 18:52:15.112921 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.112896 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-66bdc0-serving-cert\"" Oct 30 18:52:15.158435 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.158395 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699"] Oct 30 18:52:15.168884 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.168855 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mcgtf\" (UniqueName: \"kubernetes.io/projected/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-kube-api-access-mcgtf\") pod \"maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699\" (UID: \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" Oct 30 18:52:15.169038 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.168931 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-maas-api-tls\") pod \"maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699\" (UID: \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" Oct 30 18:52:15.169038 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.168956 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699\" (UID: \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" Oct 30 18:52:15.169038 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.168987 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-metrics-certs\") pod \"maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699\" (UID: \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" Oct 30 18:52:15.169210 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:52:15.169087 5894 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-serving-cert: secret "maas-api-e2e-worker-w2-66bdc0-serving-cert" not found Oct 30 18:52:15.169210 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:52:15.169167 5894 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-maas-api-tls podName:d7439fb6-eee6-483a-b43b-1f5bef9b1d49 nodeName:}" failed. No retries permitted until 2026-10-30 18:52:15.669151746 +0000 UTC m=+806.159279544 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-maas-api-tls") pod "maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" (UID: "d7439fb6-eee6-483a-b43b-1f5bef9b1d49") : secret "maas-api-e2e-worker-w2-66bdc0-serving-cert" not found Oct 30 18:52:15.169695 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.169673 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699\" (UID: \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" Oct 30 18:52:15.171409 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.171390 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-metrics-certs\") pod \"maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699\" (UID: \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" Oct 30 18:52:15.226810 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.226779 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mcgtf\" (UniqueName: \"kubernetes.io/projected/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-kube-api-access-mcgtf\") pod \"maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699\" (UID: \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" Oct 30 18:52:15.298870 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.298839 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:15.478184 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.478153 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:15.666241 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.666182 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:15.677598 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.675113 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-maas-api-tls\") pod \"maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699\" (UID: \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" Oct 30 18:52:15.695616 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.679780 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-maas-api-tls\") pod \"maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699\" (UID: \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" Oct 30 18:52:15.853551 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.853410 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:15.968259 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:15.968221 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" Oct 30 18:52:16.150974 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:16.150947 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:16.340403 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:16.340378 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699"] Oct 30 18:52:16.342957 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:52:16.342925 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7439fb6_eee6_483a_b43b_1f5bef9b1d49.slice/crio-eb2e4ae1a6f7d3183602aaaf09a8a67d5146964f648b53a769753dc7c3e92be4 WatchSource:0}: Error finding container eb2e4ae1a6f7d3183602aaaf09a8a67d5146964f648b53a769753dc7c3e92be4: Status 404 returned error can't find the container with id eb2e4ae1a6f7d3183602aaaf09a8a67d5146964f648b53a769753dc7c3e92be4 Oct 30 18:52:17.192729 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:17.192698 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" event={"ID":"d7439fb6-eee6-483a-b43b-1f5bef9b1d49","Type":"ContainerStarted","Data":"eb2e4ae1a6f7d3183602aaaf09a8a67d5146964f648b53a769753dc7c3e92be4"} Oct 30 18:52:18.179140 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:18.179112 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" Oct 30 18:52:18.199170 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:18.199135 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" event={"ID":"d7439fb6-eee6-483a-b43b-1f5bef9b1d49","Type":"ContainerStarted","Data":"2a9a1debbdbf0f4fcc7f3c7051ee51a5a84a30a0f1baed0f5a21d172b2e4d6e6"} Oct 30 18:52:18.199633 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:18.199187 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" Oct 30 18:52:18.412174 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:18.412111 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" podStartSLOduration=2.538796291 podStartE2EDuration="3.412090408s" totalImagesPullingTime="873.294117ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:18.411665868 +0000 UTC m=+808.901793692" watchObservedRunningTime="2026-10-30 18:52:18.412090408 +0000 UTC m=+808.902218229" Oct 30 18:52:20.117625 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:52:20.117563 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:52:20.127940 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.127915 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4"] Oct 30 18:52:20.131451 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.131434 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" Oct 30 18:52:20.183735 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.183707 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w1-975cf5-serving-cert\"" Oct 30 18:52:20.216926 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.216901 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb6a97e-220d-4b2b-aa99-7732009c8157-maas-api-tls\") pod \"maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4\" (UID: \"6fb6a97e-220d-4b2b-aa99-7732009c8157\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" Oct 30 18:52:20.217066 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.216932 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fb6a97e-220d-4b2b-aa99-7732009c8157-metrics-certs\") pod \"maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4\" (UID: \"6fb6a97e-220d-4b2b-aa99-7732009c8157\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" Oct 30 18:52:20.217066 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.216972 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmzdg\" (UniqueName: \"kubernetes.io/projected/6fb6a97e-220d-4b2b-aa99-7732009c8157-kube-api-access-vmzdg\") pod \"maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4\" (UID: \"6fb6a97e-220d-4b2b-aa99-7732009c8157\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" Oct 30 18:52:20.217187 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.217164 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb6a97e-220d-4b2b-aa99-7732009c8157-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4\" (UID: \"6fb6a97e-220d-4b2b-aa99-7732009c8157\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" Oct 30 18:52:20.235005 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.234984 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4"] Oct 30 18:52:20.318513 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.318476 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb6a97e-220d-4b2b-aa99-7732009c8157-maas-api-tls\") pod \"maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4\" (UID: \"6fb6a97e-220d-4b2b-aa99-7732009c8157\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" Oct 30 18:52:20.318513 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.318515 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fb6a97e-220d-4b2b-aa99-7732009c8157-metrics-certs\") pod \"maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4\" (UID: \"6fb6a97e-220d-4b2b-aa99-7732009c8157\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" Oct 30 18:52:20.318853 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.318601 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmzdg\" (UniqueName: \"kubernetes.io/projected/6fb6a97e-220d-4b2b-aa99-7732009c8157-kube-api-access-vmzdg\") pod \"maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4\" (UID: \"6fb6a97e-220d-4b2b-aa99-7732009c8157\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" Oct 30 18:52:20.318853 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.318639 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb6a97e-220d-4b2b-aa99-7732009c8157-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4\" (UID: \"6fb6a97e-220d-4b2b-aa99-7732009c8157\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" Oct 30 18:52:20.318853 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:52:20.318658 5894 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-serving-cert: secret "maas-api-e2e-worker-w1-975cf5-serving-cert" not found Oct 30 18:52:20.318853 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:52:20.318727 5894 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6fb6a97e-220d-4b2b-aa99-7732009c8157-maas-api-tls podName:6fb6a97e-220d-4b2b-aa99-7732009c8157 nodeName:}" failed. No retries permitted until 2026-10-30 18:52:20.818707871 +0000 UTC m=+811.308835687 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6fb6a97e-220d-4b2b-aa99-7732009c8157-maas-api-tls") pod "maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" (UID: "6fb6a97e-220d-4b2b-aa99-7732009c8157") : secret "maas-api-e2e-worker-w1-975cf5-serving-cert" not found Oct 30 18:52:20.319394 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.319372 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb6a97e-220d-4b2b-aa99-7732009c8157-openshift-service-ca\") pod \"maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4\" (UID: \"6fb6a97e-220d-4b2b-aa99-7732009c8157\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" Oct 30 18:52:20.321406 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.321379 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fb6a97e-220d-4b2b-aa99-7732009c8157-metrics-certs\") pod \"maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4\" (UID: \"6fb6a97e-220d-4b2b-aa99-7732009c8157\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" Oct 30 18:52:20.378300 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.378229 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmzdg\" (UniqueName: \"kubernetes.io/projected/6fb6a97e-220d-4b2b-aa99-7732009c8157-kube-api-access-vmzdg\") pod \"maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4\" (UID: \"6fb6a97e-220d-4b2b-aa99-7732009c8157\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" Oct 30 18:52:20.416925 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.416895 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:20.605998 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.605968 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:20.793275 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.793245 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:20.823031 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.823001 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb6a97e-220d-4b2b-aa99-7732009c8157-maas-api-tls\") pod \"maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4\" (UID: \"6fb6a97e-220d-4b2b-aa99-7732009c8157\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" Oct 30 18:52:20.825307 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.825288 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb6a97e-220d-4b2b-aa99-7732009c8157-maas-api-tls\") pod \"maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4\" (UID: \"6fb6a97e-220d-4b2b-aa99-7732009c8157\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" Oct 30 18:52:20.977944 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:20.977910 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:21.043923 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:21.043838 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" Oct 30 18:52:21.417427 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:21.417333 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4"] Oct 30 18:52:21.421716 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:52:21.421687 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb6a97e_220d_4b2b_aa99_7732009c8157.slice/crio-abbce4322541c0fb89f2c166e3670c9fb6296b413d2af93500e4ff8dc246797d WatchSource:0}: Error finding container abbce4322541c0fb89f2c166e3670c9fb6296b413d2af93500e4ff8dc246797d: Status 404 returned error can't find the container with id abbce4322541c0fb89f2c166e3670c9fb6296b413d2af93500e4ff8dc246797d Oct 30 18:52:22.226329 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:22.226288 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" event={"ID":"6fb6a97e-220d-4b2b-aa99-7732009c8157","Type":"ContainerStarted","Data":"abbce4322541c0fb89f2c166e3670c9fb6296b413d2af93500e4ff8dc246797d"} Oct 30 18:52:22.912355 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:22.912323 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:23.092714 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:23.092683 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:23.232020 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:23.231932 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" event={"ID":"6fb6a97e-220d-4b2b-aa99-7732009c8157","Type":"ContainerStarted","Data":"8d217fbceb28cf7f46f2df3725b22ca20742c282f6ad8b2bf552f2b59f3a2a70"} Oct 30 18:52:23.232158 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:23.232104 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" Oct 30 18:52:23.347026 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:23.346971 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" podStartSLOduration=2.470140225 podStartE2EDuration="3.346956876s" totalImagesPullingTime="876.816651ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:23.346315304 +0000 UTC m=+813.836443125" watchObservedRunningTime="2026-10-30 18:52:23.346956876 +0000 UTC m=+813.837084696" Oct 30 18:52:24.209008 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:24.208973 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" Oct 30 18:52:25.703243 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:25.703213 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:27.690785 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:27.690753 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:27.875101 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:27.875061 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:29.242848 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:29.242818 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" Oct 30 18:52:30.250320 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:30.250289 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:30.442858 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:30.442811 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:30.616061 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:30.615980 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:30.800922 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:30.800885 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:30.987892 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:30.987861 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:35.485227 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:35.485192 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:39.662227 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:39.662193 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:39.769792 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:39.769759 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq"] Oct 30 18:52:39.773440 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:39.773425 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" Oct 30 18:52:39.825872 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:39.825848 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w5-611053-serving-cert\"" Oct 30 18:52:39.877165 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:39.877135 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq"] Oct 30 18:52:39.941647 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:39.941554 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bffc276a-bd01-4f6f-a9f2-252955fa734a-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq\" (UID: \"bffc276a-bd01-4f6f-a9f2-252955fa734a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" Oct 30 18:52:39.941793 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:39.941684 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bffc276a-bd01-4f6f-a9f2-252955fa734a-metrics-certs\") pod \"maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq\" (UID: \"bffc276a-bd01-4f6f-a9f2-252955fa734a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" Oct 30 18:52:39.941793 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:39.941707 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwdz8\" (UniqueName: \"kubernetes.io/projected/bffc276a-bd01-4f6f-a9f2-252955fa734a-kube-api-access-mwdz8\") pod \"maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq\" (UID: \"bffc276a-bd01-4f6f-a9f2-252955fa734a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" Oct 30 18:52:39.941793 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:39.941741 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bffc276a-bd01-4f6f-a9f2-252955fa734a-maas-api-tls\") pod \"maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq\" (UID: \"bffc276a-bd01-4f6f-a9f2-252955fa734a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" Oct 30 18:52:40.001460 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:40.001428 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:40.043010 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:40.042980 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwdz8\" (UniqueName: \"kubernetes.io/projected/bffc276a-bd01-4f6f-a9f2-252955fa734a-kube-api-access-mwdz8\") pod \"maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq\" (UID: \"bffc276a-bd01-4f6f-a9f2-252955fa734a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" Oct 30 18:52:40.043182 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:40.043035 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bffc276a-bd01-4f6f-a9f2-252955fa734a-maas-api-tls\") pod \"maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq\" (UID: \"bffc276a-bd01-4f6f-a9f2-252955fa734a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" Oct 30 18:52:40.043182 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:40.043082 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bffc276a-bd01-4f6f-a9f2-252955fa734a-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq\" (UID: \"bffc276a-bd01-4f6f-a9f2-252955fa734a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" Oct 30 18:52:40.043182 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:40.043142 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bffc276a-bd01-4f6f-a9f2-252955fa734a-metrics-certs\") pod \"maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq\" (UID: \"bffc276a-bd01-4f6f-a9f2-252955fa734a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" Oct 30 18:52:40.043788 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:40.043764 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bffc276a-bd01-4f6f-a9f2-252955fa734a-openshift-service-ca\") pod \"maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq\" (UID: \"bffc276a-bd01-4f6f-a9f2-252955fa734a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" Oct 30 18:52:40.045412 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:40.045389 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bffc276a-bd01-4f6f-a9f2-252955fa734a-maas-api-tls\") pod \"maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq\" (UID: \"bffc276a-bd01-4f6f-a9f2-252955fa734a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" Oct 30 18:52:40.045490 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:40.045400 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bffc276a-bd01-4f6f-a9f2-252955fa734a-metrics-certs\") pod \"maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq\" (UID: \"bffc276a-bd01-4f6f-a9f2-252955fa734a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" Oct 30 18:52:40.100364 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:40.100326 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwdz8\" (UniqueName: \"kubernetes.io/projected/bffc276a-bd01-4f6f-a9f2-252955fa734a-kube-api-access-mwdz8\") pod \"maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq\" (UID: \"bffc276a-bd01-4f6f-a9f2-252955fa734a\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" Oct 30 18:52:40.184931 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:40.184895 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:40.386256 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:40.386171 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" Oct 30 18:52:40.761841 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:40.761818 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq"] Oct 30 18:52:40.764411 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:52:40.764388 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbffc276a_bd01_4f6f_a9f2_252955fa734a.slice/crio-d9d81c9e64988aa38996f915c20b623133b5316e2504c35336b829f7b4c9ac03 WatchSource:0}: Error finding container d9d81c9e64988aa38996f915c20b623133b5316e2504c35336b829f7b4c9ac03: Status 404 returned error can't find the container with id d9d81c9e64988aa38996f915c20b623133b5316e2504c35336b829f7b4c9ac03 Oct 30 18:52:41.318044 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:41.318011 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" event={"ID":"bffc276a-bd01-4f6f-a9f2-252955fa734a","Type":"ContainerStarted","Data":"d9d81c9e64988aa38996f915c20b623133b5316e2504c35336b829f7b4c9ac03"} Oct 30 18:52:42.324792 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:42.324675 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" event={"ID":"bffc276a-bd01-4f6f-a9f2-252955fa734a","Type":"ContainerStarted","Data":"8805ca0740c8dfdd3dc91e19186a9bbdad51ddcedfac96c3a080125890bcf4ea"} Oct 30 18:52:42.339027 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:42.324837 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" Oct 30 18:52:42.440710 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:42.440658 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" podStartSLOduration=2.5623505 podStartE2EDuration="3.440643252s" totalImagesPullingTime="878.292752ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:42.439789135 +0000 UTC m=+832.929916956" watchObservedRunningTime="2026-10-30 18:52:42.440643252 +0000 UTC m=+832.930771073" Oct 30 18:52:44.561362 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:44.561329 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:44.775770 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:44.775734 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz"] Oct 30 18:52:44.779530 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:44.779507 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" Oct 30 18:52:44.831733 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:44.831677 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-f5763a-serving-cert\"" Oct 30 18:52:44.882752 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:44.882722 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz"] Oct 30 18:52:44.891754 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:44.891724 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/787838ab-7e4d-423b-be79-b34816b39224-maas-api-tls\") pod \"maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz\" (UID: \"787838ab-7e4d-423b-be79-b34816b39224\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" Oct 30 18:52:44.891906 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:44.891763 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/787838ab-7e4d-423b-be79-b34816b39224-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz\" (UID: \"787838ab-7e4d-423b-be79-b34816b39224\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" Oct 30 18:52:44.891906 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:44.891884 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xnbb\" (UniqueName: \"kubernetes.io/projected/787838ab-7e4d-423b-be79-b34816b39224-kube-api-access-7xnbb\") pod \"maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz\" (UID: \"787838ab-7e4d-423b-be79-b34816b39224\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" Oct 30 18:52:44.892004 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:44.891935 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/787838ab-7e4d-423b-be79-b34816b39224-metrics-certs\") pod \"maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz\" (UID: \"787838ab-7e4d-423b-be79-b34816b39224\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" Oct 30 18:52:44.992851 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:44.992819 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/787838ab-7e4d-423b-be79-b34816b39224-metrics-certs\") pod \"maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz\" (UID: \"787838ab-7e4d-423b-be79-b34816b39224\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" Oct 30 18:52:44.993017 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:44.992884 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/787838ab-7e4d-423b-be79-b34816b39224-maas-api-tls\") pod \"maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz\" (UID: \"787838ab-7e4d-423b-be79-b34816b39224\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" Oct 30 18:52:44.993017 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:44.992907 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/787838ab-7e4d-423b-be79-b34816b39224-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz\" (UID: \"787838ab-7e4d-423b-be79-b34816b39224\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" Oct 30 18:52:44.993017 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:44.992943 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xnbb\" (UniqueName: \"kubernetes.io/projected/787838ab-7e4d-423b-be79-b34816b39224-kube-api-access-7xnbb\") pod \"maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz\" (UID: \"787838ab-7e4d-423b-be79-b34816b39224\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" Oct 30 18:52:44.993193 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:52:44.993060 5894 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-serving-cert: secret "maas-api-e2e-worker-w0-f5763a-serving-cert" not found Oct 30 18:52:44.993193 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:52:44.993137 5894 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/787838ab-7e4d-423b-be79-b34816b39224-maas-api-tls podName:787838ab-7e4d-423b-be79-b34816b39224 nodeName:}" failed. No retries permitted until 2026-10-30 18:52:45.493115499 +0000 UTC m=+835.983243315 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/787838ab-7e4d-423b-be79-b34816b39224-maas-api-tls") pod "maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" (UID: "787838ab-7e4d-423b-be79-b34816b39224") : secret "maas-api-e2e-worker-w0-f5763a-serving-cert" not found Oct 30 18:52:44.993706 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:44.993680 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/787838ab-7e4d-423b-be79-b34816b39224-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz\" (UID: \"787838ab-7e4d-423b-be79-b34816b39224\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" Oct 30 18:52:44.995198 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:44.995178 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/787838ab-7e4d-423b-be79-b34816b39224-metrics-certs\") pod \"maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz\" (UID: \"787838ab-7e4d-423b-be79-b34816b39224\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" Oct 30 18:52:45.048929 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:45.048896 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:45.050492 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:45.050468 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xnbb\" (UniqueName: \"kubernetes.io/projected/787838ab-7e4d-423b-be79-b34816b39224-kube-api-access-7xnbb\") pod \"maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz\" (UID: \"787838ab-7e4d-423b-be79-b34816b39224\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" Oct 30 18:52:45.239042 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:45.239005 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:45.499943 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:45.499859 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/787838ab-7e4d-423b-be79-b34816b39224-maas-api-tls\") pod \"maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz\" (UID: \"787838ab-7e4d-423b-be79-b34816b39224\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" Oct 30 18:52:45.500719 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:45.500697 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:45.502288 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:45.502268 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/787838ab-7e4d-423b-be79-b34816b39224-maas-api-tls\") pod \"maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz\" (UID: \"787838ab-7e4d-423b-be79-b34816b39224\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" Oct 30 18:52:45.692264 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:45.692235 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" Oct 30 18:52:46.063841 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:46.063815 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz"] Oct 30 18:52:46.066398 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:52:46.066368 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod787838ab_7e4d_423b_be79_b34816b39224.slice/crio-7f38393ae6b4f583d39e16d919c5fde995abe3a903d96faef749a963a8d54628 WatchSource:0}: Error finding container 7f38393ae6b4f583d39e16d919c5fde995abe3a903d96faef749a963a8d54628: Status 404 returned error can't find the container with id 7f38393ae6b4f583d39e16d919c5fde995abe3a903d96faef749a963a8d54628 Oct 30 18:52:46.344695 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:46.344604 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" event={"ID":"787838ab-7e4d-423b-be79-b34816b39224","Type":"ContainerStarted","Data":"7f38393ae6b4f583d39e16d919c5fde995abe3a903d96faef749a963a8d54628"} Oct 30 18:52:47.351945 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:47.351846 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" event={"ID":"787838ab-7e4d-423b-be79-b34816b39224","Type":"ContainerStarted","Data":"94da0f6be9dd79a78d31f2cf02d7d7b4dc999988a01125032b948b9aa28a965f"} Oct 30 18:52:47.352420 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:47.351961 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" Oct 30 18:52:47.466469 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:47.466406 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" podStartSLOduration=2.63239658 podStartE2EDuration="3.466390679s" totalImagesPullingTime="833.994099ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:52:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:52:47.46561501 +0000 UTC m=+837.955742827" watchObservedRunningTime="2026-10-30 18:52:47.466390679 +0000 UTC m=+837.956518496" Oct 30 18:52:48.049272 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:48.049237 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:48.334440 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:48.334354 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" Oct 30 18:52:50.117303 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:52:50.117257 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:52:50.309774 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:50.306460 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:53.362092 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:53.362059 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" Oct 30 18:52:55.337340 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:55.337301 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:52:55.507823 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:52:55.507773 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:00.390091 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:00.390047 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:00.565591 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:00.565556 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:01.075331 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:01.075300 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:01.275505 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:01.275463 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:05.630812 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:05.630777 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:08.932040 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:08.931992 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:10.790283 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:10.788865 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:10.994965 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:10.994928 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:11.617610 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:11.617558 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:11.809690 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:11.809654 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:15.681814 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:15.681781 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:16.461819 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:16.461766 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:20.117203 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:53:20.117163 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:53:20.225043 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:20.225006 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:20.632090 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:20.632056 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:20.880460 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:20.880427 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:21.103524 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:21.103486 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:25.712518 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:25.712475 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:26.637303 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:26.637265 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:27.292259 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:27.292218 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:27.518724 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:27.518688 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:30.984696 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:30.984656 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:31.960132 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:31.960098 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:32.557528 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:32.557493 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:32.766091 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:32.766050 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:33.390556 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:33.390526 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:36.263143 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:36.263105 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:40.788522 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:40.788485 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:41.146544 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:41.146496 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:42.639685 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:42.639648 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:43.283960 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:43.283908 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:43.500008 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:43.499974 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:44.350228 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.350195 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g"] Oct 30 18:53:44.354061 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.354044 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:44.365204 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.365186 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4049821e-28c6-4718-9ed9-01101012c150-tls-certs\") pod \"simulator-w4-1290a6-kserve-687975dc7b-qls8g\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:44.365301 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.365217 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-model-cache\") pod \"simulator-w4-1290a6-kserve-687975dc7b-qls8g\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:44.365377 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.365353 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kht9\" (UniqueName: \"kubernetes.io/projected/4049821e-28c6-4718-9ed9-01101012c150-kube-api-access-4kht9\") pod \"simulator-w4-1290a6-kserve-687975dc7b-qls8g\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:44.365437 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.365397 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-tmp-dir\") pod \"simulator-w4-1290a6-kserve-687975dc7b-qls8g\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:44.365515 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.365501 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-home\") pod \"simulator-w4-1290a6-kserve-687975dc7b-qls8g\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:44.365587 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.365531 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-dshm\") pod \"simulator-w4-1290a6-kserve-687975dc7b-qls8g\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:44.456206 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.456170 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-1290a6\"/\"simulator-w4-1290a6-kserve-self-signed-certs\"" Oct 30 18:53:44.456362 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.456254 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-1290a6\"/\"openshift-service-ca.crt\"" Oct 30 18:53:44.456362 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.456254 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-1290a6\"/\"default-dockercfg-5658b\"" Oct 30 18:53:44.457260 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.457245 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-1290a6\"/\"kube-root-ca.crt\"" Oct 30 18:53:44.457798 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.457779 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g"] Oct 30 18:53:44.465979 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.465959 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kht9\" (UniqueName: \"kubernetes.io/projected/4049821e-28c6-4718-9ed9-01101012c150-kube-api-access-4kht9\") pod \"simulator-w4-1290a6-kserve-687975dc7b-qls8g\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:44.465979 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.465988 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-tmp-dir\") pod \"simulator-w4-1290a6-kserve-687975dc7b-qls8g\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:44.466272 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.466028 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-home\") pod \"simulator-w4-1290a6-kserve-687975dc7b-qls8g\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:44.466272 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.466056 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-dshm\") pod \"simulator-w4-1290a6-kserve-687975dc7b-qls8g\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:44.466272 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.466113 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4049821e-28c6-4718-9ed9-01101012c150-tls-certs\") pod \"simulator-w4-1290a6-kserve-687975dc7b-qls8g\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:44.466272 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.466151 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-model-cache\") pod \"simulator-w4-1290a6-kserve-687975dc7b-qls8g\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:44.466818 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.466637 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-model-cache\") pod \"simulator-w4-1290a6-kserve-687975dc7b-qls8g\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:44.466818 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.466780 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-tmp-dir\") pod \"simulator-w4-1290a6-kserve-687975dc7b-qls8g\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:44.466966 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.466880 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-home\") pod \"simulator-w4-1290a6-kserve-687975dc7b-qls8g\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:44.468741 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.468720 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-dshm\") pod \"simulator-w4-1290a6-kserve-687975dc7b-qls8g\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:44.469044 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.469028 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4049821e-28c6-4718-9ed9-01101012c150-tls-certs\") pod \"simulator-w4-1290a6-kserve-687975dc7b-qls8g\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:44.516671 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.509463 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:44.524899 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.524864 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kht9\" (UniqueName: \"kubernetes.io/projected/4049821e-28c6-4718-9ed9-01101012c150-kube-api-access-4kht9\") pod \"simulator-w4-1290a6-kserve-687975dc7b-qls8g\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:44.665201 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:44.665161 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:45.042975 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:45.042902 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g"] Oct 30 18:53:45.046417 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:53:45.046375 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4049821e_28c6_4718_9ed9_01101012c150.slice/crio-490ab828b6f5c4ae9598ccc51e6e4aefb148788907a4ce3c474fdc05758470c8 WatchSource:0}: Error finding container 490ab828b6f5c4ae9598ccc51e6e4aefb148788907a4ce3c474fdc05758470c8: Status 404 returned error can't find the container with id 490ab828b6f5c4ae9598ccc51e6e4aefb148788907a4ce3c474fdc05758470c8 Oct 30 18:53:45.648544 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:45.648507 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" event={"ID":"4049821e-28c6-4718-9ed9-01101012c150","Type":"ContainerStarted","Data":"490ab828b6f5c4ae9598ccc51e6e4aefb148788907a4ce3c474fdc05758470c8"} Oct 30 18:53:46.220614 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:46.214469 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:47.663297 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:47.663260 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" event={"ID":"4049821e-28c6-4718-9ed9-01101012c150","Type":"ContainerStarted","Data":"b79d0b3a3aee505e588b2912cb773a8a17b81a37fc2c64e9265436646e84f531"} Oct 30 18:53:47.665427 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:47.665408 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:47.779663 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:47.779611 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podStartSLOduration=2.23366334 podStartE2EDuration="3.779568658s" totalImagesPullingTime="1.545905318s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:53:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:53:47.778838274 +0000 UTC m=+898.268966106" watchObservedRunningTime="2026-10-30 18:53:47.779568658 +0000 UTC m=+898.269696477" Oct 30 18:53:49.747530 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:49.747501 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 18:53:49.749340 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:49.749316 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 18:53:50.117511 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:53:50.117425 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:53:51.096330 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:51.096278 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:54.665971 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:54.665937 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:54.678529 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:54.678502 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:54.705868 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:54.705846 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 18:53:56.020077 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:56.020046 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:53:57.505959 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:53:57.505927 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:54:00.760265 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:00.760228 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:54:05.663612 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:05.663562 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:54:10.570503 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:10.570471 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:54:10.836337 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:10.836265 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:54:14.206443 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.206414 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l"] Oct 30 18:54:14.210144 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.210120 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:14.314005 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.313956 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-66bdc0\"/\"premium-w2-66bdc0-kserve-self-signed-certs\"" Oct 30 18:54:14.314005 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.313967 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-66bdc0\"/\"default-dockercfg-qkspm\"" Oct 30 18:54:14.314252 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.314016 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-66bdc0\"/\"openshift-service-ca.crt\"" Oct 30 18:54:14.314252 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.314024 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-66bdc0\"/\"kube-root-ca.crt\"" Oct 30 18:54:14.315763 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.315737 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l"] Oct 30 18:54:14.354856 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.354822 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:54:14.374791 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.374754 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-model-cache\") pod \"premium-w2-66bdc0-kserve-f98cf4dcf-h957l\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:14.374957 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.374797 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-tmp-dir\") pod \"premium-w2-66bdc0-kserve-f98cf4dcf-h957l\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:14.374957 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.374819 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gv84j\" (UniqueName: \"kubernetes.io/projected/3c2df19e-78b6-4db7-bc32-db2bef552a2a-kube-api-access-gv84j\") pod \"premium-w2-66bdc0-kserve-f98cf4dcf-h957l\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:14.375034 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.374995 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-dshm\") pod \"premium-w2-66bdc0-kserve-f98cf4dcf-h957l\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:14.375075 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.375026 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c2df19e-78b6-4db7-bc32-db2bef552a2a-tls-certs\") pod \"premium-w2-66bdc0-kserve-f98cf4dcf-h957l\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:14.375075 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.375052 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-home\") pod \"premium-w2-66bdc0-kserve-f98cf4dcf-h957l\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:14.476691 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.476601 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-dshm\") pod \"premium-w2-66bdc0-kserve-f98cf4dcf-h957l\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:14.476691 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.476661 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c2df19e-78b6-4db7-bc32-db2bef552a2a-tls-certs\") pod \"premium-w2-66bdc0-kserve-f98cf4dcf-h957l\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:14.476691 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.476687 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-home\") pod \"premium-w2-66bdc0-kserve-f98cf4dcf-h957l\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:14.476954 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.476750 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-model-cache\") pod \"premium-w2-66bdc0-kserve-f98cf4dcf-h957l\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:14.476954 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.476774 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-tmp-dir\") pod \"premium-w2-66bdc0-kserve-f98cf4dcf-h957l\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:14.476954 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.476901 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gv84j\" (UniqueName: \"kubernetes.io/projected/3c2df19e-78b6-4db7-bc32-db2bef552a2a-kube-api-access-gv84j\") pod \"premium-w2-66bdc0-kserve-f98cf4dcf-h957l\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:14.477351 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.477316 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-tmp-dir\") pod \"premium-w2-66bdc0-kserve-f98cf4dcf-h957l\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:14.477351 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.477335 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-model-cache\") pod \"premium-w2-66bdc0-kserve-f98cf4dcf-h957l\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:14.477531 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.477443 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-home\") pod \"premium-w2-66bdc0-kserve-f98cf4dcf-h957l\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:14.479195 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.479167 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c2df19e-78b6-4db7-bc32-db2bef552a2a-tls-certs\") pod \"premium-w2-66bdc0-kserve-f98cf4dcf-h957l\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:14.479338 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.479317 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-dshm\") pod \"premium-w2-66bdc0-kserve-f98cf4dcf-h957l\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:14.535008 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.534973 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gv84j\" (UniqueName: \"kubernetes.io/projected/3c2df19e-78b6-4db7-bc32-db2bef552a2a-kube-api-access-gv84j\") pod \"premium-w2-66bdc0-kserve-f98cf4dcf-h957l\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:14.822024 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:14.821945 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:15.203294 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:15.203269 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l"] Oct 30 18:54:15.205645 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:54:15.205618 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3c2df19e_78b6_4db7_bc32_db2bef552a2a.slice/crio-b019dabcdb46dab9f1babb8a6983201e6ea0794820ef08500c0a7932a4ca5d56 WatchSource:0}: Error finding container b019dabcdb46dab9f1babb8a6983201e6ea0794820ef08500c0a7932a4ca5d56: Status 404 returned error can't find the container with id b019dabcdb46dab9f1babb8a6983201e6ea0794820ef08500c0a7932a4ca5d56 Oct 30 18:54:15.386596 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:15.385936 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:54:15.800870 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:15.800836 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" event={"ID":"3c2df19e-78b6-4db7-bc32-db2bef552a2a","Type":"ContainerStarted","Data":"dd3c4e0e6ca81dc3c99a39612bdaf93ee9d05b2253e7879cdcc22d3a2a598e0a"} Oct 30 18:54:15.800870 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:15.800879 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" event={"ID":"3c2df19e-78b6-4db7-bc32-db2bef552a2a","Type":"ContainerStarted","Data":"b019dabcdb46dab9f1babb8a6983201e6ea0794820ef08500c0a7932a4ca5d56"} Oct 30 18:54:15.822729 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:15.822704 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:15.922154 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:15.922103 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podStartSLOduration=1.922089012 podStartE2EDuration="1.922089012s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:15.921044379 +0000 UTC m=+926.411172209" watchObservedRunningTime="2026-10-30 18:54:15.922089012 +0000 UTC m=+926.412216832" Oct 30 18:54:16.824859 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:16.824830 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:54:20.118716 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:54:20.118683 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:54:20.190100 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:20.190068 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:54:24.822768 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:24.822735 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:24.835024 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:24.834998 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:24.856425 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:24.856401 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 18:54:25.096514 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:25.096438 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:54:27.030558 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:27.030516 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:54:29.914755 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:29.914714 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:54:34.710278 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:34.710240 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:54:39.408336 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:39.408305 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:54:44.076111 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:44.076072 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:54:44.487387 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:44.487313 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:54:46.136710 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.136680 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t"] Oct 30 18:54:46.140355 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.140334 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.192883 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.192861 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w6-41db7c-openshift-default-dockercfg-fhcxz\"" Oct 30 18:54:46.244230 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.244199 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t"] Oct 30 18:54:46.328884 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.328853 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-credential-socket\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.329056 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.328901 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istiod-ca-cert\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.329056 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.328993 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjhvz\" (UniqueName: \"kubernetes.io/projected/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-kube-api-access-rjhvz\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.329056 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.329035 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-podinfo\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.329180 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.329123 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-workload-socket\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.329180 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.329156 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-data\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.329253 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.329185 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-workload-certs\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.329253 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.329207 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-token\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.329321 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.329274 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-envoy\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.430506 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.430471 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjhvz\" (UniqueName: \"kubernetes.io/projected/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-kube-api-access-rjhvz\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.430506 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.430507 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-podinfo\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.430763 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.430540 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-workload-socket\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.430763 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.430559 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-data\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.430763 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.430611 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-workload-certs\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.430763 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.430626 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-token\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.430763 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.430650 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-envoy\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.430763 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.430694 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-credential-socket\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.430763 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.430745 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istiod-ca-cert\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.431128 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.431048 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-workload-certs\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.431185 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.431140 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-data\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.431321 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.431298 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-credential-socket\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.431381 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.431339 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-workload-socket\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.431482 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.431465 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istiod-ca-cert\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.432943 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.432923 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-podinfo\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.433050 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.432970 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-envoy\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.487423 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.487387 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-token\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.487617 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.487566 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjhvz\" (UniqueName: \"kubernetes.io/projected/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-kube-api-access-rjhvz\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:46.753893 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:46.753816 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:47.233394 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:47.233369 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t"] Oct 30 18:54:47.235285 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:54:47.235262 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0464dd8d_01e7_4ef3_a866_d77fbe31d99a.slice/crio-32ad04e5b21d03b7f3dcefab7463b67c67409a95ceff489b24e9bbff282dcf4a WatchSource:0}: Error finding container 32ad04e5b21d03b7f3dcefab7463b67c67409a95ceff489b24e9bbff282dcf4a: Status 404 returned error can't find the container with id 32ad04e5b21d03b7f3dcefab7463b67c67409a95ceff489b24e9bbff282dcf4a Oct 30 18:54:47.237360 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:47.237330 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:54:47.237449 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:47.237419 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:54:47.237492 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:47.237449 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:54:47.961201 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:47.961160 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" event={"ID":"0464dd8d-01e7-4ef3-a866-d77fbe31d99a","Type":"ContainerStarted","Data":"94c54965edc3da8607376c1f49837a3b4799f6b829daef64632d841ea173af92"} Oct 30 18:54:47.961376 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:47.961223 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" event={"ID":"0464dd8d-01e7-4ef3-a866-d77fbe31d99a","Type":"ContainerStarted","Data":"32ad04e5b21d03b7f3dcefab7463b67c67409a95ceff489b24e9bbff282dcf4a"} Oct 30 18:54:48.754874 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:48.754832 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:48.756686 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:48.756660 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Oct 30 18:54:48.756789 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:48.756711 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" podUID="0464dd8d-01e7-4ef3-a866-d77fbe31d99a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Oct 30 18:54:48.856326 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:48.856114 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:54:49.754703 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:49.754674 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Oct 30 18:54:49.754904 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:49.754723 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" podUID="0464dd8d-01e7-4ef3-a866-d77fbe31d99a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Oct 30 18:54:50.117249 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:54:50.117170 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:54:50.754523 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:50.754490 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Oct 30 18:54:50.754712 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:50.754554 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" podUID="0464dd8d-01e7-4ef3-a866-d77fbe31d99a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Oct 30 18:54:51.754767 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:51.754731 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Oct 30 18:54:51.755328 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:51.754788 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" podUID="0464dd8d-01e7-4ef3-a866-d77fbe31d99a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Oct 30 18:54:52.755024 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:52.754989 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Oct 30 18:54:52.755443 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:52.755041 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" podUID="0464dd8d-01e7-4ef3-a866-d77fbe31d99a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Oct 30 18:54:53.654688 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:53.654649 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:54:53.894305 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:53.894269 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:53.895099 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:53.895079 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:53.896605 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:53.896410 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 18:54:54.068136 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:54.068035 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" podStartSLOduration=8.068020491 podStartE2EDuration="8.068020491s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:54:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:54:48.149850958 +0000 UTC m=+958.639978779" watchObservedRunningTime="2026-10-30 18:54:54.068020491 +0000 UTC m=+964.558148311" Oct 30 18:54:58.630498 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:54:58.630460 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:55:03.349117 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:03.349083 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:55:08.111369 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:08.111319 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:55:12.891350 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:12.891319 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:55:14.250729 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.250697 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2"] Oct 30 18:55:14.254199 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.254177 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.313375 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.313345 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-home\") pod \"premium-w4-1290a6-kserve-86cf84b59c-rhgn2\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.313555 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.313395 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-tls-certs\") pod \"premium-w4-1290a6-kserve-86cf84b59c-rhgn2\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.313555 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.313512 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-dshm\") pod \"premium-w4-1290a6-kserve-86cf84b59c-rhgn2\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.313555 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.313543 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-tmp-dir\") pod \"premium-w4-1290a6-kserve-86cf84b59c-rhgn2\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.313756 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.313731 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-model-cache\") pod \"premium-w4-1290a6-kserve-86cf84b59c-rhgn2\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.313820 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.313771 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4bqd\" (UniqueName: \"kubernetes.io/projected/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-kube-api-access-b4bqd\") pod \"premium-w4-1290a6-kserve-86cf84b59c-rhgn2\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.338877 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.338845 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2"] Oct 30 18:55:14.358853 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.358829 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-1290a6\"/\"premium-w4-1290a6-kserve-self-signed-certs\"" Oct 30 18:55:14.392829 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.392805 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:55:14.415305 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.415183 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-tls-certs\") pod \"premium-w4-1290a6-kserve-86cf84b59c-rhgn2\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.415305 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.415289 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-dshm\") pod \"premium-w4-1290a6-kserve-86cf84b59c-rhgn2\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.415636 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.415325 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-tmp-dir\") pod \"premium-w4-1290a6-kserve-86cf84b59c-rhgn2\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.415636 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.415445 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-model-cache\") pod \"premium-w4-1290a6-kserve-86cf84b59c-rhgn2\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.415636 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.415474 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4bqd\" (UniqueName: \"kubernetes.io/projected/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-kube-api-access-b4bqd\") pod \"premium-w4-1290a6-kserve-86cf84b59c-rhgn2\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.415636 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.415526 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-home\") pod \"premium-w4-1290a6-kserve-86cf84b59c-rhgn2\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.416111 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.416082 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-model-cache\") pod \"premium-w4-1290a6-kserve-86cf84b59c-rhgn2\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.418084 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.416313 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-tmp-dir\") pod \"premium-w4-1290a6-kserve-86cf84b59c-rhgn2\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.418961 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.416568 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-home\") pod \"premium-w4-1290a6-kserve-86cf84b59c-rhgn2\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.418961 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.418547 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-dshm\") pod \"premium-w4-1290a6-kserve-86cf84b59c-rhgn2\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.419964 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.419797 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-tls-certs\") pod \"premium-w4-1290a6-kserve-86cf84b59c-rhgn2\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.473153 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.473111 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4bqd\" (UniqueName: \"kubernetes.io/projected/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-kube-api-access-b4bqd\") pod \"premium-w4-1290a6-kserve-86cf84b59c-rhgn2\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.565672 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.565568 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:14.969875 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:14.969853 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2"] Oct 30 18:55:14.972343 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:55:14.972317 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podee442ed1_4b79_47ab_b0d9_00cc2b0d717a.slice/crio-c533e0fd2e279f627b124e06d0c5251897c6cf6a712fc74ce9410a91818ac597 WatchSource:0}: Error finding container c533e0fd2e279f627b124e06d0c5251897c6cf6a712fc74ce9410a91818ac597: Status 404 returned error can't find the container with id c533e0fd2e279f627b124e06d0c5251897c6cf6a712fc74ce9410a91818ac597 Oct 30 18:55:15.089893 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:15.089857 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" event={"ID":"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a","Type":"ContainerStarted","Data":"93ce532769e04006fc047067e33e920c2aa8976a517609e0f4672042f808ae06"} Oct 30 18:55:15.090133 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:15.089909 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" event={"ID":"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a","Type":"ContainerStarted","Data":"c533e0fd2e279f627b124e06d0c5251897c6cf6a712fc74ce9410a91818ac597"} Oct 30 18:55:15.203707 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:15.203647 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podStartSLOduration=1.203632372 podStartE2EDuration="1.203632372s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:15.203104897 +0000 UTC m=+985.693232720" watchObservedRunningTime="2026-10-30 18:55:15.203632372 +0000 UTC m=+985.693760579" Oct 30 18:55:15.566537 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:15.566501 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:17.140868 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:17.140831 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:55:17.505111 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:17.505033 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:55:20.119727 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:55:20.119690 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:55:22.186124 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:22.186065 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:55:24.566735 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:24.566700 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:24.579261 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:24.579236 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:25.168967 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:25.168937 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 18:55:25.803207 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:25.803168 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:55:26.913292 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:26.913257 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:55:29.848807 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:29.848761 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:55:31.595892 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:31.595806 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:55:36.262139 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:36.262102 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:55:36.930235 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:36.930204 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:55:41.088416 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:41.088381 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:55:45.758003 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:45.757966 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:55:47.362891 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.362858 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df"] Oct 30 18:55:47.366935 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.366911 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.419184 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.419152 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-66bdc0-openshift-default-dockercfg-d69f7\"" Oct 30 18:55:47.470693 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.470662 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df"] Oct 30 18:55:47.485037 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.485001 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-envoy\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.485266 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.485117 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kwwj\" (UniqueName: \"kubernetes.io/projected/293b1916-b37f-46c7-8ce6-b9efa6a50809-kube-api-access-7kwwj\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.485266 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.485155 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-workload-socket\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.485266 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.485194 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-workload-certs\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.485266 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.485253 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-podinfo\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.485773 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.485286 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-credential-socket\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.485862 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.485839 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-data\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.485926 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.485869 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/293b1916-b37f-46c7-8ce6-b9efa6a50809-istiod-ca-cert\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.486015 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.485987 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-token\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.587144 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.587109 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kwwj\" (UniqueName: \"kubernetes.io/projected/293b1916-b37f-46c7-8ce6-b9efa6a50809-kube-api-access-7kwwj\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.587144 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.587147 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-workload-socket\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.587366 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.587171 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-workload-certs\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.587366 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.587208 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-podinfo\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.587366 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.587230 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-credential-socket\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.587366 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.587260 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-data\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.587366 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.587283 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/293b1916-b37f-46c7-8ce6-b9efa6a50809-istiod-ca-cert\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.587366 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.587322 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-token\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.587713 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.587376 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-envoy\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.587713 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.587633 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-workload-socket\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.587839 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.587821 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-workload-certs\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.587903 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.587852 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-data\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.587903 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.587885 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-credential-socket\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.588223 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.588204 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/293b1916-b37f-46c7-8ce6-b9efa6a50809-istiod-ca-cert\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.589674 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.589654 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-envoy\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.589818 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.589799 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-podinfo\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.646750 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.646726 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-token\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.646905 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.646883 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kwwj\" (UniqueName: \"kubernetes.io/projected/293b1916-b37f-46c7-8ce6-b9efa6a50809-kube-api-access-7kwwj\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:47.679758 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:47.679730 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:48.105226 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:48.105200 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df"] Oct 30 18:55:48.107979 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:55:48.107954 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod293b1916_b37f_46c7_8ce6_b9efa6a50809.slice/crio-3ae39c9d7678a99d8d7e7eb04611baa548f87a234913f4f9651de3784e6f81c2 WatchSource:0}: Error finding container 3ae39c9d7678a99d8d7e7eb04611baa548f87a234913f4f9651de3784e6f81c2: Status 404 returned error can't find the container with id 3ae39c9d7678a99d8d7e7eb04611baa548f87a234913f4f9651de3784e6f81c2 Oct 30 18:55:48.110331 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:48.110296 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:55:48.110450 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:48.110373 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:55:48.110450 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:48.110421 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:55:48.274332 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:48.274299 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" event={"ID":"293b1916-b37f-46c7-8ce6-b9efa6a50809","Type":"ContainerStarted","Data":"0c03b664de0e6d627b50ffbf3913a252a173cf13b971f1b49bd26476f1c1bc09"} Oct 30 18:55:48.274504 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:48.274350 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" event={"ID":"293b1916-b37f-46c7-8ce6-b9efa6a50809","Type":"ContainerStarted","Data":"3ae39c9d7678a99d8d7e7eb04611baa548f87a234913f4f9651de3784e6f81c2"} Oct 30 18:55:48.394838 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:48.394781 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" podStartSLOduration=1.394762724 podStartE2EDuration="1.394762724s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:48.393541205 +0000 UTC m=+1018.883669025" watchObservedRunningTime="2026-10-30 18:55:48.394762724 +0000 UTC m=+1018.884890545" Oct 30 18:55:48.679954 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:48.679866 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:49.682289 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:49.682259 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Oct 30 18:55:49.682720 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:49.682319 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" podUID="293b1916-b37f-46c7-8ce6-b9efa6a50809" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Oct 30 18:55:50.119800 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:55:50.119716 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:55:50.680892 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:50.680855 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Oct 30 18:55:50.681093 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:50.680907 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" podUID="293b1916-b37f-46c7-8ce6-b9efa6a50809" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Oct 30 18:55:50.724880 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:50.720554 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:55:51.019661 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.019588 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8"] Oct 30 18:55:51.023429 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.023409 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.104040 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.104005 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8"] Oct 30 18:55:51.124793 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.124765 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-dshm\") pod \"premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.124911 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.124795 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-tmp-dir\") pod \"premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.124950 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.124914 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5935a99f-0b26-4651-a937-d76762e3ce55-tls-certs\") pod \"premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.125021 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.125003 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-home\") pod \"premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.125060 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.125030 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrscf\" (UniqueName: \"kubernetes.io/projected/5935a99f-0b26-4651-a937-d76762e3ce55-kube-api-access-wrscf\") pod \"premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.125123 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.125111 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-model-cache\") pod \"premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.152266 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.151937 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-bc98e1\"/\"openshift-service-ca.crt\"" Oct 30 18:55:51.152266 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.151961 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-bc98e1\"/\"premium-w3-bc98e1-kserve-self-signed-certs\"" Oct 30 18:55:51.152266 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.152066 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-bc98e1\"/\"default-dockercfg-hc99l\"" Oct 30 18:55:51.152266 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.152078 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-bc98e1\"/\"kube-root-ca.crt\"" Oct 30 18:55:51.153167 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.153137 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:55:51.226387 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.226357 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-model-cache\") pod \"premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.226589 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.226408 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-dshm\") pod \"premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.226589 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.226434 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-tmp-dir\") pod \"premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.226589 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.226513 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5935a99f-0b26-4651-a937-d76762e3ce55-tls-certs\") pod \"premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.226775 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.226608 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-home\") pod \"premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.226775 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.226636 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrscf\" (UniqueName: \"kubernetes.io/projected/5935a99f-0b26-4651-a937-d76762e3ce55-kube-api-access-wrscf\") pod \"premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.230597 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.226930 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-model-cache\") pod \"premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.235596 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.234871 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-home\") pod \"premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.235596 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.235408 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5935a99f-0b26-4651-a937-d76762e3ce55-tls-certs\") pod \"premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.235775 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.235667 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-tmp-dir\") pod \"premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.238377 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.238338 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-dshm\") pod \"premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.287149 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.287076 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrscf\" (UniqueName: \"kubernetes.io/projected/5935a99f-0b26-4651-a937-d76762e3ce55-kube-api-access-wrscf\") pod \"premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.354495 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.354453 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:51.680819 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.680783 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Oct 30 18:55:51.681004 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.680836 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" podUID="293b1916-b37f-46c7-8ce6-b9efa6a50809" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Oct 30 18:55:51.803019 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:51.802984 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8"] Oct 30 18:55:51.805341 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:55:51.805312 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5935a99f_0b26_4651_a937_d76762e3ce55.slice/crio-ac4a3922c9dfacfa65193aae8723dcdefd98d9498cc934e1551228816bc7119b WatchSource:0}: Error finding container ac4a3922c9dfacfa65193aae8723dcdefd98d9498cc934e1551228816bc7119b: Status 404 returned error can't find the container with id ac4a3922c9dfacfa65193aae8723dcdefd98d9498cc934e1551228816bc7119b Oct 30 18:55:52.298033 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:52.297995 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" event={"ID":"5935a99f-0b26-4651-a937-d76762e3ce55","Type":"ContainerStarted","Data":"c73922259a69a8a99a39c4820af9e0853c6bdea474a2880cd6249744d30b9231"} Oct 30 18:55:52.298189 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:52.298041 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" event={"ID":"5935a99f-0b26-4651-a937-d76762e3ce55","Type":"ContainerStarted","Data":"ac4a3922c9dfacfa65193aae8723dcdefd98d9498cc934e1551228816bc7119b"} Oct 30 18:55:52.355714 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:52.355675 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:55:52.413805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:52.413755 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podStartSLOduration=2.413742697 podStartE2EDuration="2.413742697s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:55:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:55:52.41218132 +0000 UTC m=+1022.902309141" watchObservedRunningTime="2026-10-30 18:55:52.413742697 +0000 UTC m=+1022.903870548" Oct 30 18:55:52.681096 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:52.681061 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Oct 30 18:55:52.681292 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:52.681108 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" podUID="293b1916-b37f-46c7-8ce6-b9efa6a50809" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Oct 30 18:55:53.680192 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:53.680164 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Oct 30 18:55:53.680567 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:53.680207 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" podUID="293b1916-b37f-46c7-8ce6-b9efa6a50809" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Oct 30 18:55:54.827980 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:54.827948 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:54.829155 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:54.829129 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:54.830076 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:54.830053 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 18:55:55.480651 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:55:55.472943 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:56:00.275742 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:00.275703 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:56:01.355453 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:01.355419 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:56:01.368009 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:01.367984 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:56:02.360060 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:02.360031 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 18:56:04.961850 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:04.961802 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:56:09.781518 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:09.781483 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:56:14.410005 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:14.409964 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:56:17.563083 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:17.563054 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:56:19.129091 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:19.129053 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:56:20.117302 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:56:20.117266 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:56:23.863768 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:23.863723 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:56:28.674978 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:28.674936 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:56:33.361324 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:33.361289 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:56:34.442675 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:34.442641 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:56:38.085256 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:38.085201 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:56:42.805991 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:42.805937 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:56:46.186652 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:46.186563 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:56:46.362599 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:46.362546 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:56:47.045906 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.045873 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs"] Oct 30 18:56:47.049737 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.049716 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.142992 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.142959 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-workload-socket\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.142992 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.142995 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-workload-certs\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.143192 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.143112 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-data\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.143192 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.143166 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-credential-socket\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.143274 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.143194 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-envoy\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.143274 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.143214 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-podinfo\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.143274 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.143234 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4b4e21b-a660-4b77-869c-64ec2a9996be-istiod-ca-cert\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.143380 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.143339 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbb78\" (UniqueName: \"kubernetes.io/projected/e4b4e21b-a660-4b77-869c-64ec2a9996be-kube-api-access-jbb78\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.143418 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.143398 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-token\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.153647 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.153623 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs"] Oct 30 18:56:47.244235 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.244201 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-workload-socket\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.244677 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.244239 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-workload-certs\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.244677 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.244291 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-data\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.244677 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.244329 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-credential-socket\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.244677 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.244369 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-envoy\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.244677 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.244408 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-podinfo\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.244985 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.244959 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-workload-socket\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.244985 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.244956 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4b4e21b-a660-4b77-869c-64ec2a9996be-istiod-ca-cert\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.245112 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.245067 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbb78\" (UniqueName: \"kubernetes.io/projected/e4b4e21b-a660-4b77-869c-64ec2a9996be-kube-api-access-jbb78\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.245112 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.244963 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-workload-certs\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.245218 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.245196 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-token\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.245417 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.245068 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-data\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.245746 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.245704 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4b4e21b-a660-4b77-869c-64ec2a9996be-istiod-ca-cert\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.245746 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.245719 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-credential-socket\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.253708 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.247512 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-envoy\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.253708 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.247638 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-podinfo\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.304531 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.304459 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-token\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.304778 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.304751 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbb78\" (UniqueName: \"kubernetes.io/projected/e4b4e21b-a660-4b77-869c-64ec2a9996be-kube-api-access-jbb78\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.363355 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.363322 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:47.585150 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.585044 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:56:47.792592 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.792546 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs"] Oct 30 18:56:47.796117 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:56:47.796091 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4b4e21b_a660_4b77_869c_64ec2a9996be.slice/crio-d0ab90d0a96fecbb1c09f45dcc703383e9ee7e79bd7e48e19f5000fde95c5542 WatchSource:0}: Error finding container d0ab90d0a96fecbb1c09f45dcc703383e9ee7e79bd7e48e19f5000fde95c5542: Status 404 returned error can't find the container with id d0ab90d0a96fecbb1c09f45dcc703383e9ee7e79bd7e48e19f5000fde95c5542 Oct 30 18:56:47.798362 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.798329 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:56:47.798449 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.798394 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:56:47.798449 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:47.798423 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:56:48.341967 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.341933 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr"] Oct 30 18:56:48.345805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.345782 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.398391 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.398360 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w5-611053-openshift-default-dockercfg-7h2nn\"" Oct 30 18:56:48.450256 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.450219 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr"] Oct 30 18:56:48.457850 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.457823 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istiod-ca-cert\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.457987 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.457860 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-podinfo\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.457987 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.457888 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-workload-socket\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.458079 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.457994 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-token\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.458079 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.458024 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-envoy\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.458178 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.458157 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-data\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.458227 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.458211 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-credential-socket\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.458266 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.458235 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-workload-certs\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.458266 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.458260 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vqqg7\" (UniqueName: \"kubernetes.io/projected/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-kube-api-access-vqqg7\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.561194 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.561144 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-credential-socket\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.562002 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.561978 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-workload-certs\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.562544 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.562526 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vqqg7\" (UniqueName: \"kubernetes.io/projected/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-kube-api-access-vqqg7\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.563095 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.563073 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istiod-ca-cert\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.563920 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.563902 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-podinfo\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.564475 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.564456 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-workload-socket\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.564634 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.564615 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-token\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.564776 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.564762 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-envoy\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.564989 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.564974 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-data\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.569959 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.562450 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-workload-certs\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.569959 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.566093 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-workload-socket\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.569959 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.563840 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istiod-ca-cert\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.569959 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.561914 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-credential-socket\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.570875 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.570844 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-data\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.572719 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.572697 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-envoy\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.572965 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.572947 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-podinfo\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.612692 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.612632 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" event={"ID":"e4b4e21b-a660-4b77-869c-64ec2a9996be","Type":"ContainerStarted","Data":"dc633cfa9302d94206772134950af8f1055edcabf0ee8a9813db39b5cec4f282"} Oct 30 18:56:48.612692 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.612680 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" event={"ID":"e4b4e21b-a660-4b77-869c-64ec2a9996be","Type":"ContainerStarted","Data":"d0ab90d0a96fecbb1c09f45dcc703383e9ee7e79bd7e48e19f5000fde95c5542"} Oct 30 18:56:48.620001 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.619980 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vqqg7\" (UniqueName: \"kubernetes.io/projected/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-kube-api-access-vqqg7\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.622602 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.622566 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-token\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:48.659386 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:48.659356 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:49.083096 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:49.083043 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" podStartSLOduration=2.083028735 podStartE2EDuration="2.083028735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:48.754109445 +0000 UTC m=+1079.244237266" watchObservedRunningTime="2026-10-30 18:56:49.083028735 +0000 UTC m=+1079.573156560" Oct 30 18:56:49.084327 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:49.084309 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr"] Oct 30 18:56:49.086624 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:56:49.086597 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0133d032_d0fb_4623_8c0e_4ddc8b765c6d.slice/crio-9761171fbd0846fc8fd5017723935b2f7b7c8cc9cfa53e4b2ff4d3f0dc075b8d WatchSource:0}: Error finding container 9761171fbd0846fc8fd5017723935b2f7b7c8cc9cfa53e4b2ff4d3f0dc075b8d: Status 404 returned error can't find the container with id 9761171fbd0846fc8fd5017723935b2f7b7c8cc9cfa53e4b2ff4d3f0dc075b8d Oct 30 18:56:49.088897 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:49.088872 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:56:49.088964 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:49.088930 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:56:49.088999 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:49.088978 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:56:49.364858 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:49.364773 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:49.365954 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:49.365929 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Oct 30 18:56:49.366085 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:49.365981 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" podUID="e4b4e21b-a660-4b77-869c-64ec2a9996be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Oct 30 18:56:49.621061 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:49.620962 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" event={"ID":"0133d032-d0fb-4623-8c0e-4ddc8b765c6d","Type":"ContainerStarted","Data":"5941388e9c1d6ca368b3b98ca036b6bdf8980a374e924171797f95fe1850d341"} Oct 30 18:56:49.621061 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:49.621032 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" event={"ID":"0133d032-d0fb-4623-8c0e-4ddc8b765c6d","Type":"ContainerStarted","Data":"9761171fbd0846fc8fd5017723935b2f7b7c8cc9cfa53e4b2ff4d3f0dc075b8d"} Oct 30 18:56:49.665609 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:49.665246 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:49.760600 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:49.760203 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" podStartSLOduration=1.760184643 podStartE2EDuration="1.760184643s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:56:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:56:49.74904956 +0000 UTC m=+1080.239177380" watchObservedRunningTime="2026-10-30 18:56:49.760184643 +0000 UTC m=+1080.250312464" Oct 30 18:56:50.124010 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:56:50.123956 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:56:50.365598 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:50.364917 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Oct 30 18:56:50.365598 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:50.364976 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" podUID="e4b4e21b-a660-4b77-869c-64ec2a9996be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Oct 30 18:56:50.664550 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:50.664510 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 30 18:56:50.664862 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:50.664831 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" podUID="0133d032-d0fb-4623-8c0e-4ddc8b765c6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 30 18:56:51.364985 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:51.364936 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Oct 30 18:56:51.365299 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:51.365252 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" podUID="e4b4e21b-a660-4b77-869c-64ec2a9996be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Oct 30 18:56:51.676739 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:51.676279 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 30 18:56:51.676739 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:51.676338 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" podUID="0133d032-d0fb-4623-8c0e-4ddc8b765c6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 30 18:56:52.366598 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:52.365957 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Oct 30 18:56:52.366598 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:52.366021 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" podUID="e4b4e21b-a660-4b77-869c-64ec2a9996be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Oct 30 18:56:52.394150 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:52.394123 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:56:52.670748 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:52.670709 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 30 18:56:52.670996 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:52.670966 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" podUID="0133d032-d0fb-4623-8c0e-4ddc8b765c6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 30 18:56:53.364894 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:53.364857 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Oct 30 18:56:53.365496 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:53.365464 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" podUID="e4b4e21b-a660-4b77-869c-64ec2a9996be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Oct 30 18:56:53.670599 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:53.661965 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 30 18:56:53.670599 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:53.662026 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" podUID="0133d032-d0fb-4623-8c0e-4ddc8b765c6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 30 18:56:54.363955 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:54.363909 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Oct 30 18:56:54.364164 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:54.363988 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" podUID="e4b4e21b-a660-4b77-869c-64ec2a9996be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Oct 30 18:56:54.660799 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:54.660765 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" start-of-body= Oct 30 18:56:54.661217 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:54.660830 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" podUID="0133d032-d0fb-4623-8c0e-4ddc8b765c6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": dial tcp 10.132.0.61:15021: connect: connection refused" Oct 30 18:56:55.368009 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:55.367979 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:55.368450 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:55.368428 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:55.369021 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:55.368999 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 18:56:55.909904 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:55.909872 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:55.910450 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:55.910420 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:55.911161 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:55.911140 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 18:56:57.159964 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:57.159928 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:56:58.378465 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:56:58.378425 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:57:01.855762 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:01.855507 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh"] Oct 30 18:57:01.862676 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:01.862626 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:01.916933 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:01.916901 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-1290a6-openshift-default-dockercfg-zvdjm\"" Oct 30 18:57:01.961608 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:01.957652 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-workload-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:01.961608 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:01.957732 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-credential-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:01.961608 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:01.957867 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-workload-certs\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:01.961608 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:01.957903 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc2hh\" (UniqueName: \"kubernetes.io/projected/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-kube-api-access-nc2hh\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:01.961608 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:01.957943 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-token\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:01.961608 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:01.957980 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-envoy\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:01.961608 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:01.958033 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-data\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:01.961608 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:01.958083 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-podinfo\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:01.961608 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:01.958113 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istiod-ca-cert\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:01.962342 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:01.961762 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:57:02.013681 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.012679 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh"] Oct 30 18:57:02.062729 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.062181 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-workload-certs\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:02.062729 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.062242 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc2hh\" (UniqueName: \"kubernetes.io/projected/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-kube-api-access-nc2hh\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:02.062729 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.062271 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-token\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:02.062729 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.062309 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-envoy\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:02.062729 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.062345 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-data\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:02.062729 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.062377 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-podinfo\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:02.062729 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.062409 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istiod-ca-cert\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:02.062729 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.062511 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-workload-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:02.062729 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.062547 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-credential-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:02.063690 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.063091 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-credential-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:02.063690 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.063361 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-workload-certs\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:02.102004 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.101969 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istiod-ca-cert\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:02.102507 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.102479 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-workload-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:02.102906 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.102883 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-envoy\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:02.103312 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.103289 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-data\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:02.106843 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.106788 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-podinfo\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:02.121680 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.121651 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-token\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:02.123489 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.123461 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc2hh\" (UniqueName: \"kubernetes.io/projected/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-kube-api-access-nc2hh\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:02.207919 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.207813 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:02.699324 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.699297 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh"] Oct 30 18:57:02.702176 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:57:02.702150 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1d373aa_3cfa_4f6a_9746_889e0d90cde2.slice/crio-6982fdd8e093acc11f4c91714eb8008fa88fefe1fd71f951aeb13f655eb74929 WatchSource:0}: Error finding container 6982fdd8e093acc11f4c91714eb8008fa88fefe1fd71f951aeb13f655eb74929: Status 404 returned error can't find the container with id 6982fdd8e093acc11f4c91714eb8008fa88fefe1fd71f951aeb13f655eb74929 Oct 30 18:57:02.704490 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.704458 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:57:02.704568 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.704521 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:57:02.704568 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.704551 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:57:02.763533 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.763508 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" event={"ID":"d1d373aa-3cfa-4f6a-9746-889e0d90cde2","Type":"ContainerStarted","Data":"6982fdd8e093acc11f4c91714eb8008fa88fefe1fd71f951aeb13f655eb74929"} Oct 30 18:57:02.958633 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.958540 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl"] Oct 30 18:57:02.962530 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:02.962500 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.026592 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.026541 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl"] Oct 30 18:57:03.109708 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.109672 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-data\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.109894 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.109748 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-podinfo\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.109894 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.109783 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-workload-socket\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.109894 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.109871 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-envoy\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.110068 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.109938 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e867658-2d43-4edc-9d26-f23fdd765a80-istiod-ca-cert\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.110068 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.109969 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-credential-socket\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.110068 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.110045 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-workload-certs\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.110212 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.110085 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-token\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.110212 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.110105 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67lp9\" (UniqueName: \"kubernetes.io/projected/9e867658-2d43-4edc-9d26-f23fdd765a80-kube-api-access-67lp9\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.210973 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.210887 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e867658-2d43-4edc-9d26-f23fdd765a80-istiod-ca-cert\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.210973 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.210930 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-credential-socket\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.210973 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.210969 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-workload-certs\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.211214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.211004 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-token\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.211214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.211023 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67lp9\" (UniqueName: \"kubernetes.io/projected/9e867658-2d43-4edc-9d26-f23fdd765a80-kube-api-access-67lp9\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.211214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.211052 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-data\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.211214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.211075 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-podinfo\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.211214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.211109 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-workload-socket\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.211214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.211156 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-envoy\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.211467 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.211441 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-workload-certs\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.211543 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.211521 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-credential-socket\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.211543 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.211533 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-data\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.211691 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.211673 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-workload-socket\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.211816 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.211797 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e867658-2d43-4edc-9d26-f23fdd765a80-istiod-ca-cert\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.213377 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.213358 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-envoy\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.213738 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.213720 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-podinfo\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.269905 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.269874 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-token\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.270078 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.269993 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67lp9\" (UniqueName: \"kubernetes.io/projected/9e867658-2d43-4edc-9d26-f23fdd765a80-kube-api-access-67lp9\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.276787 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.276758 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:03.721562 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.721538 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl"] Oct 30 18:57:03.723666 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:57:03.723632 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e867658_2d43_4edc_9d26_f23fdd765a80.slice/crio-7221e1b8d3a5766a115503b17f969cf2c1dab570f93da9ba8bfbb7fa0d4f9084 WatchSource:0}: Error finding container 7221e1b8d3a5766a115503b17f969cf2c1dab570f93da9ba8bfbb7fa0d4f9084: Status 404 returned error can't find the container with id 7221e1b8d3a5766a115503b17f969cf2c1dab570f93da9ba8bfbb7fa0d4f9084 Oct 30 18:57:03.726556 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.726526 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:57:03.726715 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.726649 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:57:03.726715 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.726695 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:57:03.779649 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.776890 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" event={"ID":"9e867658-2d43-4edc-9d26-f23fdd765a80","Type":"ContainerStarted","Data":"7221e1b8d3a5766a115503b17f969cf2c1dab570f93da9ba8bfbb7fa0d4f9084"} Oct 30 18:57:03.779649 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.778477 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" event={"ID":"d1d373aa-3cfa-4f6a-9746-889e0d90cde2","Type":"ContainerStarted","Data":"f9e63c22c869e759a80c0049dae21694907a8535ee8c63eb75a927ac6e22f99d"} Oct 30 18:57:03.904323 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:03.904265 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" podStartSLOduration=2.904249961 podStartE2EDuration="2.904249961s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:03.900565883 +0000 UTC m=+1094.390693703" watchObservedRunningTime="2026-10-30 18:57:03.904249961 +0000 UTC m=+1094.394377781" Oct 30 18:57:04.208346 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:04.208313 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:04.209795 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:04.209766 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Oct 30 18:57:04.209927 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:04.209825 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" podUID="d1d373aa-3cfa-4f6a-9746-889e0d90cde2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Oct 30 18:57:04.786623 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:04.786559 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" event={"ID":"9e867658-2d43-4edc-9d26-f23fdd765a80","Type":"ContainerStarted","Data":"29edc299346db07618b499a54e63bd4392058106256f2828ce551469ff899996"} Oct 30 18:57:04.967609 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:04.964961 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" podStartSLOduration=2.964939747 podStartE2EDuration="2.964939747s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:04.960150236 +0000 UTC m=+1095.450278061" watchObservedRunningTime="2026-10-30 18:57:04.964939747 +0000 UTC m=+1095.455067573" Oct 30 18:57:05.208558 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:05.208529 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Oct 30 18:57:05.209010 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:05.208982 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" podUID="d1d373aa-3cfa-4f6a-9746-889e0d90cde2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Oct 30 18:57:05.278685 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:05.278638 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:05.278949 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:05.278920 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Oct 30 18:57:05.279050 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:05.278980 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" podUID="9e867658-2d43-4edc-9d26-f23fdd765a80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Oct 30 18:57:06.208806 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:06.208771 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Oct 30 18:57:06.209188 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:06.208828 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" podUID="d1d373aa-3cfa-4f6a-9746-889e0d90cde2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Oct 30 18:57:06.280023 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:06.279977 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Oct 30 18:57:06.280221 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:06.280031 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" podUID="9e867658-2d43-4edc-9d26-f23fdd765a80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Oct 30 18:57:06.848601 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:06.848540 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:57:07.208495 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:07.208464 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Oct 30 18:57:07.208703 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:07.208515 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" podUID="d1d373aa-3cfa-4f6a-9746-889e0d90cde2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Oct 30 18:57:07.278596 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:07.278126 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Oct 30 18:57:07.278596 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:07.278198 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" podUID="9e867658-2d43-4edc-9d26-f23fdd765a80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Oct 30 18:57:08.208829 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:08.208797 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Oct 30 18:57:08.208998 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:08.208850 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" podUID="d1d373aa-3cfa-4f6a-9746-889e0d90cde2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Oct 30 18:57:08.277960 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:08.277892 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Oct 30 18:57:08.277960 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:08.277940 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" podUID="9e867658-2d43-4edc-9d26-f23fdd765a80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Oct 30 18:57:09.208433 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:09.208402 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" start-of-body= Oct 30 18:57:09.208433 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:09.208461 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" podUID="d1d373aa-3cfa-4f6a-9746-889e0d90cde2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.62:15021/healthz/ready\": dial tcp 10.132.0.62:15021: connect: connection refused" Oct 30 18:57:09.278145 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:09.278103 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Oct 30 18:57:09.278347 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:09.278153 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" podUID="9e867658-2d43-4edc-9d26-f23fdd765a80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Oct 30 18:57:10.213492 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:10.213462 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:10.213901 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:10.213771 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:10.214381 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:10.214364 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 18:57:10.277930 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:10.277902 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" start-of-body= Oct 30 18:57:10.278064 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:10.277977 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" podUID="9e867658-2d43-4edc-9d26-f23fdd765a80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.63:15021/healthz/ready\": dial tcp 10.132.0.63:15021: connect: connection refused" Oct 30 18:57:11.281839 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:11.281807 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:11.282320 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:11.282178 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:11.282894 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:11.282872 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 18:57:11.766015 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:11.765986 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:57:16.459120 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:16.459087 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:57:17.883115 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:17.883087 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:57:20.119161 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:57:20.119126 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:57:21.151792 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:21.151733 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:57:25.994734 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:25.994686 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:57:30.610266 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:30.610232 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:57:35.257299 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:35.257270 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:57:39.968897 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:39.968858 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:57:41.459973 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:41.459934 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:57:44.604704 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:44.604664 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:57:44.786177 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:44.786144 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:57:49.476042 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:49.475962 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:57:50.127669 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:57:50.127627 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:57:54.205957 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:54.205886 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:57:54.957884 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:54.957852 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l"] Oct 30 18:57:54.961754 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:54.961737 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.036331 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.036297 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-model-cache\") pod \"distinct-w6-41db7c-kserve-5fc49465d4-b9g7l\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.036509 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.036336 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-dshm\") pod \"distinct-w6-41db7c-kserve-5fc49465d4-b9g7l\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.036509 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.036365 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-tmp-dir\") pod \"distinct-w6-41db7c-kserve-5fc49465d4-b9g7l\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.036509 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.036482 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjhxm\" (UniqueName: \"kubernetes.io/projected/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-kube-api-access-jjhxm\") pod \"distinct-w6-41db7c-kserve-5fc49465d4-b9g7l\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.036736 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.036511 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-tls-certs\") pod \"distinct-w6-41db7c-kserve-5fc49465d4-b9g7l\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.036736 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.036530 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-home\") pod \"distinct-w6-41db7c-kserve-5fc49465d4-b9g7l\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.065791 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.065757 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-41db7c\"/\"openshift-service-ca.crt\"" Oct 30 18:57:55.065944 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.065856 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w6-41db7c\"/\"kube-root-ca.crt\"" Oct 30 18:57:55.065944 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.065898 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-41db7c\"/\"distinct-w6-41db7c-kserve-self-signed-certs\"" Oct 30 18:57:55.066065 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.065991 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w6-41db7c\"/\"default-dockercfg-46k5f\"" Oct 30 18:57:55.067973 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.067946 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l"] Oct 30 18:57:55.093120 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.093088 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:57:55.144138 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.140153 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-tmp-dir\") pod \"distinct-w6-41db7c-kserve-5fc49465d4-b9g7l\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.144138 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.140237 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjhxm\" (UniqueName: \"kubernetes.io/projected/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-kube-api-access-jjhxm\") pod \"distinct-w6-41db7c-kserve-5fc49465d4-b9g7l\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.144138 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.140273 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-tls-certs\") pod \"distinct-w6-41db7c-kserve-5fc49465d4-b9g7l\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.144138 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.140305 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-home\") pod \"distinct-w6-41db7c-kserve-5fc49465d4-b9g7l\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.144138 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.140477 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-model-cache\") pod \"distinct-w6-41db7c-kserve-5fc49465d4-b9g7l\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.144138 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.140506 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-dshm\") pod \"distinct-w6-41db7c-kserve-5fc49465d4-b9g7l\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.160083 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.148634 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-dshm\") pod \"distinct-w6-41db7c-kserve-5fc49465d4-b9g7l\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.160083 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.148931 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-home\") pod \"distinct-w6-41db7c-kserve-5fc49465d4-b9g7l\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.160083 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.149166 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-model-cache\") pod \"distinct-w6-41db7c-kserve-5fc49465d4-b9g7l\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.160083 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.149468 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-tmp-dir\") pod \"distinct-w6-41db7c-kserve-5fc49465d4-b9g7l\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.160083 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.149600 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-tls-certs\") pod \"distinct-w6-41db7c-kserve-5fc49465d4-b9g7l\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.206360 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.206333 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjhxm\" (UniqueName: \"kubernetes.io/projected/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-kube-api-access-jjhxm\") pod \"distinct-w6-41db7c-kserve-5fc49465d4-b9g7l\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.273958 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.273868 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:55.673057 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:55.672992 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l"] Oct 30 18:57:55.676190 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:57:55.676144 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod67c4ee9e_c041_4f48_936b_afb5a69dbd1f.slice/crio-d32598ac3378af5683bb4b65e4a4b21991c8f8919361e4fb9c7ed7422fdab91b WatchSource:0}: Error finding container d32598ac3378af5683bb4b65e4a4b21991c8f8919361e4fb9c7ed7422fdab91b: Status 404 returned error can't find the container with id d32598ac3378af5683bb4b65e4a4b21991c8f8919361e4fb9c7ed7422fdab91b Oct 30 18:57:56.096803 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:56.096719 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" event={"ID":"67c4ee9e-c041-4f48-936b-afb5a69dbd1f","Type":"ContainerStarted","Data":"ac4f072bc9807e52550b3c997115da13b99601b954c6c8b6484cffe0c396b189"} Oct 30 18:57:56.096803 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:56.096758 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" event={"ID":"67c4ee9e-c041-4f48-936b-afb5a69dbd1f","Type":"ContainerStarted","Data":"d32598ac3378af5683bb4b65e4a4b21991c8f8919361e4fb9c7ed7422fdab91b"} Oct 30 18:57:56.214807 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:56.214752 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podStartSLOduration=2.214737562 podStartE2EDuration="2.214737562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:57:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:57:56.212685143 +0000 UTC m=+1146.702812965" watchObservedRunningTime="2026-10-30 18:57:56.214737562 +0000 UTC m=+1146.704865444" Oct 30 18:57:56.274757 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:56.274731 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:57:57.278319 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:57.278285 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6"] Oct 30 18:57:57.278798 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:57.278511 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" podUID="187e78e5-00ef-4ee9-97a7-4b6fa262c551" containerName="istio-proxy" containerID="cri-o://95c9810d8b818ec4c75c785f1de172c159729d0f54b735a79bf24390065e32e5" gracePeriod=30 Oct 30 18:57:59.010606 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:57:59.008774 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:58:02.537628 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.537598 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:58:02.647330 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.647298 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-envoy\") pod \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " Oct 30 18:58:02.647524 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.647344 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-workload-certs\") pod \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " Oct 30 18:58:02.647524 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.647418 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istiod-ca-cert\") pod \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " Oct 30 18:58:02.647524 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.647447 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-credential-socket\") pod \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " Oct 30 18:58:02.647524 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.647473 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-data\") pod \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " Oct 30 18:58:02.647524 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.647500 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-token\" (UniqueName: \"kubernetes.io/projected/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-token\") pod \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " Oct 30 18:58:02.647807 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.647598 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/187e78e5-00ef-4ee9-97a7-4b6fa262c551-kube-api-access-mk8s8\" (UniqueName: \"kubernetes.io/projected/187e78e5-00ef-4ee9-97a7-4b6fa262c551-kube-api-access-mk8s8\") pod \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " Oct 30 18:58:02.647807 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.647633 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-workload-certs" pod "187e78e5-00ef-4ee9-97a7-4b6fa262c551" (UID: "187e78e5-00ef-4ee9-97a7-4b6fa262c551"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:58:02.647807 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.647711 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-podinfo\") pod \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " Oct 30 18:58:02.647807 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.647763 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-workload-socket\") pod \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\" (UID: \"187e78e5-00ef-4ee9-97a7-4b6fa262c551\") " Oct 30 18:58:02.648036 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.647799 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-data" pod "187e78e5-00ef-4ee9-97a7-4b6fa262c551" (UID: "187e78e5-00ef-4ee9-97a7-4b6fa262c551"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:58:02.648036 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.647877 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-credential-socket" pod "187e78e5-00ef-4ee9-97a7-4b6fa262c551" (UID: "187e78e5-00ef-4ee9-97a7-4b6fa262c551"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:58:02.648138 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.648065 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-workload-socket" pod "187e78e5-00ef-4ee9-97a7-4b6fa262c551" (UID: "187e78e5-00ef-4ee9-97a7-4b6fa262c551"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:58:02.648138 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.648089 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istiod-ca-cert" pod "187e78e5-00ef-4ee9-97a7-4b6fa262c551" (UID: "187e78e5-00ef-4ee9-97a7-4b6fa262c551"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:58:02.648355 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.648181 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:58:02.648355 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.648198 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:58:02.648355 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.648210 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:58:02.648355 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.648222 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:58:02.649820 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.649793 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-podinfo" pod "187e78e5-00ef-4ee9-97a7-4b6fa262c551" (UID: "187e78e5-00ef-4ee9-97a7-4b6fa262c551"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 18:58:02.649919 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.649798 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/187e78e5-00ef-4ee9-97a7-4b6fa262c551-kube-api-access-mk8s8" pod "187e78e5-00ef-4ee9-97a7-4b6fa262c551" (UID: "187e78e5-00ef-4ee9-97a7-4b6fa262c551"). InnerVolumeSpecName "kube-api-access-mk8s8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:58:02.649973 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.649919 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-envoy" pod "187e78e5-00ef-4ee9-97a7-4b6fa262c551" (UID: "187e78e5-00ef-4ee9-97a7-4b6fa262c551"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:58:02.650042 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.650016 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-token" pod "187e78e5-00ef-4ee9-97a7-4b6fa262c551" (UID: "187e78e5-00ef-4ee9-97a7-4b6fa262c551"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:58:02.749251 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.749218 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:58:02.749251 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.749249 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:58:02.749450 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.749262 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:58:02.749450 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.749274 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/187e78e5-00ef-4ee9-97a7-4b6fa262c551-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:58:02.749450 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:02.749287 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mk8s8\" (UniqueName: \"kubernetes.io/projected/187e78e5-00ef-4ee9-97a7-4b6fa262c551-kube-api-access-mk8s8\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:58:03.134596 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:03.134494 5894 generic.go:356] "Generic (PLEG): container finished" podID="187e78e5-00ef-4ee9-97a7-4b6fa262c551" containerID="95c9810d8b818ec4c75c785f1de172c159729d0f54b735a79bf24390065e32e5" exitCode=0 Oct 30 18:58:03.134596 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:03.134547 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" Oct 30 18:58:03.134596 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:03.134590 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" event={"ID":"187e78e5-00ef-4ee9-97a7-4b6fa262c551","Type":"ContainerDied","Data":"95c9810d8b818ec4c75c785f1de172c159729d0f54b735a79bf24390065e32e5"} Oct 30 18:58:03.134814 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:03.134632 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6" event={"ID":"187e78e5-00ef-4ee9-97a7-4b6fa262c551","Type":"ContainerDied","Data":"5650964eaffb1dd910c9ba2b19c0836f20c04390728caf866ff5e3155eba2ad3"} Oct 30 18:58:03.134814 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:03.134653 5894 scope.go:122] "RemoveContainer" containerID="95c9810d8b818ec4c75c785f1de172c159729d0f54b735a79bf24390065e32e5" Oct 30 18:58:03.144597 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:03.144565 5894 scope.go:122] "RemoveContainer" containerID="95c9810d8b818ec4c75c785f1de172c159729d0f54b735a79bf24390065e32e5" Oct 30 18:58:03.144892 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:58:03.144872 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95c9810d8b818ec4c75c785f1de172c159729d0f54b735a79bf24390065e32e5\": container with ID starting with 95c9810d8b818ec4c75c785f1de172c159729d0f54b735a79bf24390065e32e5 not found: ID does not exist" containerID="95c9810d8b818ec4c75c785f1de172c159729d0f54b735a79bf24390065e32e5" Oct 30 18:58:03.144942 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:03.144904 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95c9810d8b818ec4c75c785f1de172c159729d0f54b735a79bf24390065e32e5"} err="failed to get container status \"95c9810d8b818ec4c75c785f1de172c159729d0f54b735a79bf24390065e32e5\": rpc error: code = NotFound desc = could not find container \"95c9810d8b818ec4c75c785f1de172c159729d0f54b735a79bf24390065e32e5\": container with ID starting with 95c9810d8b818ec4c75c785f1de172c159729d0f54b735a79bf24390065e32e5 not found: ID does not exist" Oct 30 18:58:03.308853 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:03.308818 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6"] Oct 30 18:58:03.359138 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:03.359100 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-mvch6"] Oct 30 18:58:03.649872 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:03.649841 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="187e78e5-00ef-4ee9-97a7-4b6fa262c551" path="/var/lib/kubelet/pods/187e78e5-00ef-4ee9-97a7-4b6fa262c551/volumes" Oct 30 18:58:03.681274 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:03.681245 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:58:05.274805 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:05.274776 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:58:05.287867 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:05.287838 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:58:06.163237 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:06.163206 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 18:58:08.432940 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:08.432901 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:58:13.151329 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:13.151251 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:58:13.410821 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:13.410730 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:58:13.592514 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:13.592479 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:58:18.071151 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:18.071111 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:58:18.247998 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:18.247962 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:58:20.120850 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:58:20.120814 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:58:22.887660 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:22.887629 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:58:25.497345 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:25.497283 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:58:27.784294 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:27.784216 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:58:31.158733 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:31.158650 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:58:32.478974 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:32.478583 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:58:34.425325 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.425287 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps"] Oct 30 18:58:34.425968 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.425951 5894 cpu_manager.go:414] "containerMap: removing container" podUID="187e78e5-00ef-4ee9-97a7-4b6fa262c551" containerName="istio-proxy" Oct 30 18:58:34.426013 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.425970 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="187e78e5-00ef-4ee9-97a7-4b6fa262c551" containerName="istio-proxy" Oct 30 18:58:34.426072 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.426063 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="187e78e5-00ef-4ee9-97a7-4b6fa262c551" containerName="istio-proxy" Oct 30 18:58:34.429824 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.429807 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.535224 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.535187 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-workload-certs\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.535386 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.535259 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-podinfo\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.535386 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.535345 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-workload-socket\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.535386 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.535373 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-token\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.535515 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.535474 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-bc98e1-openshift-default-dockercfg-mrm2g\"" Oct 30 18:58:34.535555 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.535536 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-data\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.535616 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.535599 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-envoy\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.535674 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.535658 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-credential-socket\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.535739 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.535723 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istiod-ca-cert\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.535781 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.535751 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vmsm\" (UniqueName: \"kubernetes.io/projected/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-kube-api-access-9vmsm\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.537665 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.537647 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps"] Oct 30 18:58:34.637319 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.637286 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-podinfo\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.637492 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.637335 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-workload-socket\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.637492 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.637351 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-token\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.637492 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.637416 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-data\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.637665 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.637631 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-envoy\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.637734 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.637717 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-credential-socket\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.637795 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.637727 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-workload-socket\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.637795 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.637751 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istiod-ca-cert\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.637795 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.637784 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vmsm\" (UniqueName: \"kubernetes.io/projected/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-kube-api-access-9vmsm\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.637939 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.637813 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-workload-certs\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.637939 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.637786 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-data\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.638050 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.638027 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-credential-socket\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.638160 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.638144 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-workload-certs\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.638363 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.638342 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istiod-ca-cert\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.640228 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.640203 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-envoy\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.640431 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.640414 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-podinfo\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.695223 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.695150 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-token\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.695453 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.695426 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vmsm\" (UniqueName: \"kubernetes.io/projected/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-kube-api-access-9vmsm\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:34.744319 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:34.744289 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:35.169502 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:35.169476 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps"] Oct 30 18:58:35.171541 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:58:35.171509 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62f6a40a_acbe_4f4b_98c9_6a3ed3ecad66.slice/crio-f3fedc5bccf2c949134bdc95b710884918527e73d5b63d7a7e4828ee409bf826 WatchSource:0}: Error finding container f3fedc5bccf2c949134bdc95b710884918527e73d5b63d7a7e4828ee409bf826: Status 404 returned error can't find the container with id f3fedc5bccf2c949134bdc95b710884918527e73d5b63d7a7e4828ee409bf826 Oct 30 18:58:35.173746 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:35.173716 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:58:35.173817 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:35.173779 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:58:35.173855 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:35.173815 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:58:35.336214 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:35.336178 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" event={"ID":"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66","Type":"ContainerStarted","Data":"582f6fa875c3e25dfca56050790e94b17539fa081567e399f1177034a8170a8b"} Oct 30 18:58:35.336402 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:35.336245 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" event={"ID":"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66","Type":"ContainerStarted","Data":"f3fedc5bccf2c949134bdc95b710884918527e73d5b63d7a7e4828ee409bf826"} Oct 30 18:58:35.455567 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:35.455463 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" podStartSLOduration=1.455439069 podStartE2EDuration="1.455439069s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:35.453473353 +0000 UTC m=+1185.943601173" watchObservedRunningTime="2026-10-30 18:58:35.455439069 +0000 UTC m=+1185.945566888" Oct 30 18:58:35.745653 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:35.744663 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:36.752788 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:36.752750 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Oct 30 18:58:36.753259 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:36.752808 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" podUID="62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Oct 30 18:58:37.183963 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:37.182690 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:58:37.749598 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:37.749112 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Oct 30 18:58:37.749598 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:37.749168 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" podUID="62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Oct 30 18:58:38.752018 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:38.751978 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Oct 30 18:58:38.752408 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:38.752032 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" podUID="62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Oct 30 18:58:39.750597 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:39.749898 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Oct 30 18:58:39.750597 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:39.749954 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" podUID="62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Oct 30 18:58:40.752033 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:40.751993 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Oct 30 18:58:40.752421 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:40.752051 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" podUID="62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Oct 30 18:58:41.745248 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:41.745217 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" start-of-body= Oct 30 18:58:41.745414 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:41.745270 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" podUID="62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.65:15021/healthz/ready\": dial tcp 10.132.0.65:15021: connect: connection refused" Oct 30 18:58:41.961825 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:41.961331 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:58:42.749893 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:42.749864 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:42.750124 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:42.750103 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:42.750768 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:42.750749 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 18:58:46.617302 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:46.617266 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:58:47.871235 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:47.871202 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/periodic-gathering-cp4ws-pwgzh"] Oct 30 18:58:47.875112 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:47.875096 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" Oct 30 18:58:47.931026 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:47.930990 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-cp4ws-pwgzh"] Oct 30 18:58:48.001985 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.001951 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/20536d00-75e4-4f08-acd3-02b64cd659c0-archives-path\") pod \"periodic-gathering-cp4ws-pwgzh\" (UID: \"20536d00-75e4-4f08-acd3-02b64cd659c0\") " pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" Oct 30 18:58:48.002156 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.001996 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20536d00-75e4-4f08-acd3-02b64cd659c0-trusted-ca-bundle\") pod \"periodic-gathering-cp4ws-pwgzh\" (UID: \"20536d00-75e4-4f08-acd3-02b64cd659c0\") " pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" Oct 30 18:58:48.002156 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.002113 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdz9q\" (UniqueName: \"kubernetes.io/projected/20536d00-75e4-4f08-acd3-02b64cd659c0-kube-api-access-zdz9q\") pod \"periodic-gathering-cp4ws-pwgzh\" (UID: \"20536d00-75e4-4f08-acd3-02b64cd659c0\") " pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" Oct 30 18:58:48.002306 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.002292 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20536d00-75e4-4f08-acd3-02b64cd659c0-service-ca-bundle\") pod \"periodic-gathering-cp4ws-pwgzh\" (UID: \"20536d00-75e4-4f08-acd3-02b64cd659c0\") " pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" Oct 30 18:58:48.103563 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.103522 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20536d00-75e4-4f08-acd3-02b64cd659c0-service-ca-bundle\") pod \"periodic-gathering-cp4ws-pwgzh\" (UID: \"20536d00-75e4-4f08-acd3-02b64cd659c0\") " pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" Oct 30 18:58:48.104113 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.103630 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/20536d00-75e4-4f08-acd3-02b64cd659c0-archives-path\") pod \"periodic-gathering-cp4ws-pwgzh\" (UID: \"20536d00-75e4-4f08-acd3-02b64cd659c0\") " pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" Oct 30 18:58:48.104113 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.103663 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20536d00-75e4-4f08-acd3-02b64cd659c0-trusted-ca-bundle\") pod \"periodic-gathering-cp4ws-pwgzh\" (UID: \"20536d00-75e4-4f08-acd3-02b64cd659c0\") " pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" Oct 30 18:58:48.104113 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.103706 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdz9q\" (UniqueName: \"kubernetes.io/projected/20536d00-75e4-4f08-acd3-02b64cd659c0-kube-api-access-zdz9q\") pod \"periodic-gathering-cp4ws-pwgzh\" (UID: \"20536d00-75e4-4f08-acd3-02b64cd659c0\") " pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" Oct 30 18:58:48.104306 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.104257 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/20536d00-75e4-4f08-acd3-02b64cd659c0-archives-path\") pod \"periodic-gathering-cp4ws-pwgzh\" (UID: \"20536d00-75e4-4f08-acd3-02b64cd659c0\") " pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" Oct 30 18:58:48.104406 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.104386 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20536d00-75e4-4f08-acd3-02b64cd659c0-service-ca-bundle\") pod \"periodic-gathering-cp4ws-pwgzh\" (UID: \"20536d00-75e4-4f08-acd3-02b64cd659c0\") " pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" Oct 30 18:58:48.104899 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.104879 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20536d00-75e4-4f08-acd3-02b64cd659c0-trusted-ca-bundle\") pod \"periodic-gathering-cp4ws-pwgzh\" (UID: \"20536d00-75e4-4f08-acd3-02b64cd659c0\") " pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" Oct 30 18:58:48.161588 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.161548 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdz9q\" (UniqueName: \"kubernetes.io/projected/20536d00-75e4-4f08-acd3-02b64cd659c0-kube-api-access-zdz9q\") pod \"periodic-gathering-cp4ws-pwgzh\" (UID: \"20536d00-75e4-4f08-acd3-02b64cd659c0\") " pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" Oct 30 18:58:48.188360 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.188334 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" Oct 30 18:58:48.298663 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.298634 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5"] Oct 30 18:58:48.302839 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.302819 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.355213 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.355179 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-f5763a-openshift-default-dockercfg-hrzwg\"" Oct 30 18:58:48.403625 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.403591 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5"] Oct 30 18:58:48.406365 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.406335 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-token\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.406491 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.406382 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-credential-socket\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.406491 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.406406 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-envoy\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.406569 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.406546 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbj9m\" (UniqueName: \"kubernetes.io/projected/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-kube-api-access-rbj9m\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.406623 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.406590 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-podinfo\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.406784 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.406768 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-workload-socket\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.406827 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.406793 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istiod-ca-cert\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.406876 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.406830 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-data\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.406953 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.406936 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-workload-certs\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.508088 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.508008 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-workload-socket\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.508088 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.508043 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istiod-ca-cert\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.508088 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.508060 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-data\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.508353 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.508097 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-workload-certs\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.508353 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.508129 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-token\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.508353 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.508159 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-credential-socket\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.508353 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.508176 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-envoy\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.508353 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.508238 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbj9m\" (UniqueName: \"kubernetes.io/projected/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-kube-api-access-rbj9m\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.508353 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.508264 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-podinfo\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.508685 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.508637 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-workload-socket\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.508685 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.508670 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-data\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.508835 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.508816 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-credential-socket\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.508905 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.508883 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-workload-certs\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.509100 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.509076 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istiod-ca-cert\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.510664 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.510644 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-podinfo\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.510745 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.510676 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-envoy\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.561029 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.561003 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/periodic-gathering-cp4ws-pwgzh"] Oct 30 18:58:48.563039 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:58:48.563011 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20536d00_75e4_4f08_acd3_02b64cd659c0.slice/crio-70ea5cf7b087ffc9d0681f39b06d9f16ae71410e17af385dbf30d8de28494173 WatchSource:0}: Error finding container 70ea5cf7b087ffc9d0681f39b06d9f16ae71410e17af385dbf30d8de28494173: Status 404 returned error can't find the container with id 70ea5cf7b087ffc9d0681f39b06d9f16ae71410e17af385dbf30d8de28494173 Oct 30 18:58:48.565618 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.565598 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-token\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.566566 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.566545 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbj9m\" (UniqueName: \"kubernetes.io/projected/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-kube-api-access-rbj9m\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:48.616074 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:48.616052 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:49.045987 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:49.045963 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5"] Oct 30 18:58:49.047530 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:58:49.047501 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7acd06a_8759_4b0a_9e4e_7874e9fd38e5.slice/crio-c4b54f84f6cf9424afc0e0d1dfb117c05652bc257aad6991ec917f6eeb22c577 WatchSource:0}: Error finding container c4b54f84f6cf9424afc0e0d1dfb117c05652bc257aad6991ec917f6eeb22c577: Status 404 returned error can't find the container with id c4b54f84f6cf9424afc0e0d1dfb117c05652bc257aad6991ec917f6eeb22c577 Oct 30 18:58:49.049872 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:49.049842 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:58:49.049950 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:49.049903 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:58:49.049950 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:49.049931 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:58:49.439247 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:49.439207 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" event={"ID":"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5","Type":"ContainerStarted","Data":"1c6cbfb9bdc0ecee30c44d556524a839b0d371f5e7200ddae3cb9371b931499c"} Oct 30 18:58:49.439434 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:49.439273 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" event={"ID":"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5","Type":"ContainerStarted","Data":"c4b54f84f6cf9424afc0e0d1dfb117c05652bc257aad6991ec917f6eeb22c577"} Oct 30 18:58:49.440766 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:49.440731 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" event={"ID":"20536d00-75e4-4f08-acd3-02b64cd659c0","Type":"ContainerStarted","Data":"fd7aa2800175708e077e112aade52134e05250ebd6d79eca962a324e4f5c5ba7"} Oct 30 18:58:49.440903 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:49.440776 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" event={"ID":"20536d00-75e4-4f08-acd3-02b64cd659c0","Type":"ContainerStarted","Data":"70ea5cf7b087ffc9d0681f39b06d9f16ae71410e17af385dbf30d8de28494173"} Oct 30 18:58:49.557131 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:49.557076 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" podStartSLOduration=1.557062089 podStartE2EDuration="1.557062089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:49.555147696 +0000 UTC m=+1200.045275516" watchObservedRunningTime="2026-10-30 18:58:49.557062089 +0000 UTC m=+1200.047189908" Oct 30 18:58:49.617098 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:49.617068 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:49.670103 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:49.670041 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" podStartSLOduration=2.670023262 podStartE2EDuration="2.670023262s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:58:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:58:49.669281146 +0000 UTC m=+1200.159408966" watchObservedRunningTime="2026-10-30 18:58:49.670023262 +0000 UTC m=+1200.160151082" Oct 30 18:58:49.786684 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:49.786615 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 18:58:49.787363 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:49.787341 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 18:58:50.122369 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:58:50.122295 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:58:50.618209 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:50.618175 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 18:58:50.618375 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:50.618232 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" podUID="c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 18:58:51.515837 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:51.507959 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:58:51.616731 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:51.616697 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 18:58:51.616902 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:51.616747 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" podUID="c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 18:58:52.616595 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:52.616545 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 18:58:52.616980 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:52.616618 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" podUID="c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 18:58:53.616714 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:53.616681 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 18:58:53.617103 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:53.616730 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" podUID="c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 18:58:54.616471 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:54.616436 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 18:58:54.616638 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:54.616491 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" podUID="c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 18:58:55.616905 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:55.616862 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" start-of-body= Oct 30 18:58:55.617279 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:55.616932 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" podUID="c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.67:15021/healthz/ready\": dial tcp 10.132.0.67:15021: connect: connection refused" Oct 30 18:58:56.363888 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:56.363857 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:58:56.875479 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:56.875447 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:56.876857 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:56.876831 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:58:56.877550 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:58:56.877530 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 18:59:01.185059 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:01.185024 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:02.128120 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:02.128048 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:05.910684 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:05.910633 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:09.377992 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:09.377960 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:10.832601 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:10.832538 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:12.916540 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:12.916508 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr"] Oct 30 18:59:12.916979 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:12.916710 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" podUID="b18383d6-7b02-4191-80de-440dc8bbee34" containerName="istio-proxy" containerID="cri-o://f260126e58c6d54ff2019510ad08236ff6f17c4fccdd09cbc87e627e3ccb0d4d" gracePeriod=30 Oct 30 18:59:13.425747 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.425713 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn"] Oct 30 18:59:13.429787 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.429766 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:13.487046 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.486954 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-611053\"/\"distinct-2-w5-611053-kserve-self-signed-certs\"" Oct 30 18:59:13.490435 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.490401 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn"] Oct 30 18:59:13.536291 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.536267 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w5-611053\"/\"default-dockercfg-h72q7\"" Oct 30 18:59:13.536460 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.536437 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-611053\"/\"kube-root-ca.crt\"" Oct 30 18:59:13.536533 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.536518 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w5-611053\"/\"openshift-service-ca.crt\"" Oct 30 18:59:13.540064 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.540044 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-dshm\") pod \"distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:13.540123 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.540080 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-home\") pod \"distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:13.540123 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.540101 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-model-cache\") pod \"distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:13.540272 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.540257 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhz7w\" (UniqueName: \"kubernetes.io/projected/4c72bcc9-7401-4b86-900d-558b768d8d16-kube-api-access-lhz7w\") pod \"distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:13.540321 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.540308 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c72bcc9-7401-4b86-900d-558b768d8d16-tls-certs\") pod \"distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:13.540426 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.540409 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-tmp-dir\") pod \"distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:13.595230 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.595188 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:13.642126 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.642079 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhz7w\" (UniqueName: \"kubernetes.io/projected/4c72bcc9-7401-4b86-900d-558b768d8d16-kube-api-access-lhz7w\") pod \"distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:13.642316 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.642181 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c72bcc9-7401-4b86-900d-558b768d8d16-tls-certs\") pod \"distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:13.642316 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.642240 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-tmp-dir\") pod \"distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:13.642442 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.642417 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-dshm\") pod \"distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:13.642535 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.642477 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-home\") pod \"distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:13.642535 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.642524 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-model-cache\") pod \"distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:13.642778 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.642750 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-tmp-dir\") pod \"distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:13.642900 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.642858 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-home\") pod \"distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:13.642994 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.642973 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-model-cache\") pod \"distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:13.644872 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.644847 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-dshm\") pod \"distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:13.645402 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.645379 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c72bcc9-7401-4b86-900d-558b768d8d16-tls-certs\") pod \"distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:13.700623 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.700592 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhz7w\" (UniqueName: \"kubernetes.io/projected/4c72bcc9-7401-4b86-900d-558b768d8d16-kube-api-access-lhz7w\") pod \"distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:13.742352 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:13.742270 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:14.229106 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:14.229082 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn"] Oct 30 18:59:14.232867 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:59:14.232832 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4c72bcc9_7401_4b86_900d_558b768d8d16.slice/crio-2640ea3a8603b1b65e86f6a5d534452e45a603fdcad05e7ce911f1cbed69c808 WatchSource:0}: Error finding container 2640ea3a8603b1b65e86f6a5d534452e45a603fdcad05e7ce911f1cbed69c808: Status 404 returned error can't find the container with id 2640ea3a8603b1b65e86f6a5d534452e45a603fdcad05e7ce911f1cbed69c808 Oct 30 18:59:14.613425 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:14.613352 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" event={"ID":"4c72bcc9-7401-4b86-900d-558b768d8d16","Type":"ContainerStarted","Data":"22872da387db898771bf377c0134d75a87b8744dc5c42a8d8479c4eacdc5333a"} Oct 30 18:59:14.613425 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:14.613395 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" event={"ID":"4c72bcc9-7401-4b86-900d-558b768d8d16","Type":"ContainerStarted","Data":"2640ea3a8603b1b65e86f6a5d534452e45a603fdcad05e7ce911f1cbed69c808"} Oct 30 18:59:14.737981 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:14.737929 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podStartSLOduration=1.737915366 podStartE2EDuration="1.737915366s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:14.736177379 +0000 UTC m=+1225.226305200" watchObservedRunningTime="2026-10-30 18:59:14.737915366 +0000 UTC m=+1225.228043180" Oct 30 18:59:14.743017 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:14.742996 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:15.601755 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:15.599380 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:18.250709 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.250677 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:59:18.329080 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.329039 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b18383d6-7b02-4191-80de-440dc8bbee34-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18383d6-7b02-4191-80de-440dc8bbee34-istiod-ca-cert\") pod \"b18383d6-7b02-4191-80de-440dc8bbee34\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " Oct 30 18:59:18.329357 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.329336 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-istio-data\") pod \"b18383d6-7b02-4191-80de-440dc8bbee34\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " Oct 30 18:59:18.329507 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.329490 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b18383d6-7b02-4191-80de-440dc8bbee34-istio-token\" (UniqueName: \"kubernetes.io/projected/b18383d6-7b02-4191-80de-440dc8bbee34-istio-token\") pod \"b18383d6-7b02-4191-80de-440dc8bbee34\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " Oct 30 18:59:18.329693 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.329674 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-credential-socket\") pod \"b18383d6-7b02-4191-80de-440dc8bbee34\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " Oct 30 18:59:18.329835 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.329819 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-workload-certs\") pod \"b18383d6-7b02-4191-80de-440dc8bbee34\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " Oct 30 18:59:18.329947 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.329931 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-istio-envoy\") pod \"b18383d6-7b02-4191-80de-440dc8bbee34\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " Oct 30 18:59:18.330070 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.330055 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b18383d6-7b02-4191-80de-440dc8bbee34-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18383d6-7b02-4191-80de-440dc8bbee34-istio-podinfo\") pod \"b18383d6-7b02-4191-80de-440dc8bbee34\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " Oct 30 18:59:18.330357 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.330151 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-istio-data" pod "b18383d6-7b02-4191-80de-440dc8bbee34" (UID: "b18383d6-7b02-4191-80de-440dc8bbee34"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:59:18.330357 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.330189 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b18383d6-7b02-4191-80de-440dc8bbee34-kube-api-access-bkgl6\" (UniqueName: \"kubernetes.io/projected/b18383d6-7b02-4191-80de-440dc8bbee34-kube-api-access-bkgl6\") pod \"b18383d6-7b02-4191-80de-440dc8bbee34\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " Oct 30 18:59:18.330357 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.330231 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-workload-socket\") pod \"b18383d6-7b02-4191-80de-440dc8bbee34\" (UID: \"b18383d6-7b02-4191-80de-440dc8bbee34\") " Oct 30 18:59:18.330854 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.330724 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-workload-certs" pod "b18383d6-7b02-4191-80de-440dc8bbee34" (UID: "b18383d6-7b02-4191-80de-440dc8bbee34"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:59:18.332278 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.332230 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b18383d6-7b02-4191-80de-440dc8bbee34-istiod-ca-cert" pod "b18383d6-7b02-4191-80de-440dc8bbee34" (UID: "b18383d6-7b02-4191-80de-440dc8bbee34"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 18:59:18.335681 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.335651 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-credential-socket" pod "b18383d6-7b02-4191-80de-440dc8bbee34" (UID: "b18383d6-7b02-4191-80de-440dc8bbee34"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:59:18.336562 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.336531 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-workload-socket" pod "b18383d6-7b02-4191-80de-440dc8bbee34" (UID: "b18383d6-7b02-4191-80de-440dc8bbee34"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:59:18.336712 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.336689 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:59:18.339267 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.336718 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:59:18.339267 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.336732 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:59:18.339267 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.336746 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:59:18.342963 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.342927 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-istio-envoy" pod "b18383d6-7b02-4191-80de-440dc8bbee34" (UID: "b18383d6-7b02-4191-80de-440dc8bbee34"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 18:59:18.355732 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.349669 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b18383d6-7b02-4191-80de-440dc8bbee34-kube-api-access-bkgl6" pod "b18383d6-7b02-4191-80de-440dc8bbee34" (UID: "b18383d6-7b02-4191-80de-440dc8bbee34"). InnerVolumeSpecName "kube-api-access-bkgl6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:59:18.357493 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.357467 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b18383d6-7b02-4191-80de-440dc8bbee34-istio-podinfo" pod "b18383d6-7b02-4191-80de-440dc8bbee34" (UID: "b18383d6-7b02-4191-80de-440dc8bbee34"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 18:59:18.360729 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.360705 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b18383d6-7b02-4191-80de-440dc8bbee34-istio-token" pod "b18383d6-7b02-4191-80de-440dc8bbee34" (UID: "b18383d6-7b02-4191-80de-440dc8bbee34"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 18:59:18.437798 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.437761 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bkgl6\" (UniqueName: \"kubernetes.io/projected/b18383d6-7b02-4191-80de-440dc8bbee34-kube-api-access-bkgl6\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:59:18.437798 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.437799 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b18383d6-7b02-4191-80de-440dc8bbee34-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:59:18.437988 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.437815 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b18383d6-7b02-4191-80de-440dc8bbee34-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:59:18.437988 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.437828 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b18383d6-7b02-4191-80de-440dc8bbee34-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:59:18.437988 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.437842 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b18383d6-7b02-4191-80de-440dc8bbee34-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 18:59:18.579687 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.579583 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:18.668553 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.668523 5894 generic.go:356] "Generic (PLEG): container finished" podID="b18383d6-7b02-4191-80de-440dc8bbee34" containerID="f260126e58c6d54ff2019510ad08236ff6f17c4fccdd09cbc87e627e3ccb0d4d" exitCode=0 Oct 30 18:59:18.668743 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.668589 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" Oct 30 18:59:18.668743 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.668607 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" event={"ID":"b18383d6-7b02-4191-80de-440dc8bbee34","Type":"ContainerDied","Data":"f260126e58c6d54ff2019510ad08236ff6f17c4fccdd09cbc87e627e3ccb0d4d"} Oct 30 18:59:18.668743 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.668666 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr" event={"ID":"b18383d6-7b02-4191-80de-440dc8bbee34","Type":"ContainerDied","Data":"2f97e0ed97f0c9e94acddafae4246005ab483766e10d5af5db1aeff6c0050c80"} Oct 30 18:59:18.668743 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.668692 5894 scope.go:122] "RemoveContainer" containerID="f260126e58c6d54ff2019510ad08236ff6f17c4fccdd09cbc87e627e3ccb0d4d" Oct 30 18:59:18.679125 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.679107 5894 scope.go:122] "RemoveContainer" containerID="f260126e58c6d54ff2019510ad08236ff6f17c4fccdd09cbc87e627e3ccb0d4d" Oct 30 18:59:18.679466 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:59:18.679439 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f260126e58c6d54ff2019510ad08236ff6f17c4fccdd09cbc87e627e3ccb0d4d\": container with ID starting with f260126e58c6d54ff2019510ad08236ff6f17c4fccdd09cbc87e627e3ccb0d4d not found: ID does not exist" containerID="f260126e58c6d54ff2019510ad08236ff6f17c4fccdd09cbc87e627e3ccb0d4d" Oct 30 18:59:18.679519 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.679482 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f260126e58c6d54ff2019510ad08236ff6f17c4fccdd09cbc87e627e3ccb0d4d"} err="failed to get container status \"f260126e58c6d54ff2019510ad08236ff6f17c4fccdd09cbc87e627e3ccb0d4d\": rpc error: code = NotFound desc = could not find container \"f260126e58c6d54ff2019510ad08236ff6f17c4fccdd09cbc87e627e3ccb0d4d\": container with ID starting with f260126e58c6d54ff2019510ad08236ff6f17c4fccdd09cbc87e627e3ccb0d4d not found: ID does not exist" Oct 30 18:59:18.834329 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.834248 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr"] Oct 30 18:59:18.883635 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:18.883604 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2lmdr"] Oct 30 18:59:19.650826 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:19.650783 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b18383d6-7b02-4191-80de-440dc8bbee34" path="/var/lib/kubelet/pods/b18383d6-7b02-4191-80de-440dc8bbee34/volumes" Oct 30 18:59:20.117819 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:59:20.117743 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:59:20.374448 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:20.374377 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:22.981932 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:22.981903 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:23.586294 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.586262 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6"] Oct 30 18:59:23.586860 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.586845 5894 cpu_manager.go:414] "containerMap: removing container" podUID="b18383d6-7b02-4191-80de-440dc8bbee34" containerName="istio-proxy" Oct 30 18:59:23.586914 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.586862 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b18383d6-7b02-4191-80de-440dc8bbee34" containerName="istio-proxy" Oct 30 18:59:23.586959 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.586950 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="b18383d6-7b02-4191-80de-440dc8bbee34" containerName="istio-proxy" Oct 30 18:59:23.591727 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.591705 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:23.694095 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.694054 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-bc98e1\"/\"distinct-2-w3-bc98e1-kserve-self-signed-certs\"" Oct 30 18:59:23.696523 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.696492 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6"] Oct 30 18:59:23.718096 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.718064 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-home\") pod \"distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:23.718251 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.718110 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlsz9\" (UniqueName: \"kubernetes.io/projected/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-kube-api-access-rlsz9\") pod \"distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:23.718319 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.718259 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-tls-certs\") pod \"distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:23.718359 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.718341 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-dshm\") pod \"distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:23.718414 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.718365 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-model-cache\") pod \"distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:23.718414 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.718398 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-tmp-dir\") pod \"distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:23.725076 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.725045 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:23.745470 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.745436 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:23.770511 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.770482 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:23.820163 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.820107 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-dshm\") pod \"distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:23.820360 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.820178 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-model-cache\") pod \"distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:23.820360 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.820232 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-tmp-dir\") pod \"distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:23.820482 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.820366 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-home\") pod \"distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:23.820482 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.820425 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlsz9\" (UniqueName: \"kubernetes.io/projected/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-kube-api-access-rlsz9\") pod \"distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:23.820605 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.820516 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-tls-certs\") pod \"distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:23.820918 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.820892 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-model-cache\") pod \"distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:23.821039 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.820941 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-home\") pod \"distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:23.821162 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.821142 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-tmp-dir\") pod \"distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:23.823614 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.823591 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-dshm\") pod \"distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:23.823993 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.823958 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-tls-certs\") pod \"distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:23.877732 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.877665 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlsz9\" (UniqueName: \"kubernetes.io/projected/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-kube-api-access-rlsz9\") pod \"distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:23.904383 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:23.904355 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:24.323731 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:24.323704 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6"] Oct 30 18:59:24.326331 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:59:24.326295 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode1e51799_7dbd_47ed_9b66_e4e355cc03b7.slice/crio-96c128daca2cab662799714dce29bdaefd496c68939165289f0c123f83b75664 WatchSource:0}: Error finding container 96c128daca2cab662799714dce29bdaefd496c68939165289f0c123f83b75664: Status 404 returned error can't find the container with id 96c128daca2cab662799714dce29bdaefd496c68939165289f0c123f83b75664 Oct 30 18:59:24.705737 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:24.705701 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" event={"ID":"e1e51799-7dbd-47ed-9b66-e4e355cc03b7","Type":"ContainerStarted","Data":"31d0600e78d1ec2f7e7be1e683fe284fbe0f7b6a64e08ec6b4e99dc8a5df39a1"} Oct 30 18:59:24.705894 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:24.705748 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" event={"ID":"e1e51799-7dbd-47ed-9b66-e4e355cc03b7","Type":"ContainerStarted","Data":"96c128daca2cab662799714dce29bdaefd496c68939165289f0c123f83b75664"} Oct 30 18:59:24.717585 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:24.717558 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 18:59:24.819970 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:24.819917 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podStartSLOduration=1.819902049 podStartE2EDuration="1.819902049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:24.818785897 +0000 UTC m=+1235.308913711" watchObservedRunningTime="2026-10-30 18:59:24.819902049 +0000 UTC m=+1235.310029868" Oct 30 18:59:24.904867 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:24.904841 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:25.270619 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:25.259606 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:29.919601 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:29.915694 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:33.021081 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.021047 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv"] Oct 30 18:59:33.024947 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.024928 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.129652 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.129623 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv"] Oct 30 18:59:33.129831 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.129760 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-workload-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.129904 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.129837 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-podinfo\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.129959 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.129904 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-workload-certs\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.129959 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.129929 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istiod-ca-cert\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.130067 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.130035 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mn58g\" (UniqueName: \"kubernetes.io/projected/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-kube-api-access-mn58g\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.130129 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.130068 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-credential-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.130129 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.130093 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-envoy\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.130245 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.130131 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-data\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.130245 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.130157 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-token\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.231006 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.230974 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-workload-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.231157 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.231033 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-podinfo\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.231157 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.231073 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-workload-certs\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.231157 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.231097 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istiod-ca-cert\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.231323 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.231172 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mn58g\" (UniqueName: \"kubernetes.io/projected/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-kube-api-access-mn58g\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.231323 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.231223 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-credential-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.231418 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.231379 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-envoy\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.231474 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.231422 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-workload-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.231474 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.231429 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-data\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.231609 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.231496 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-token\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.231696 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.231668 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-workload-certs\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.231825 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.231803 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-data\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.231911 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.231892 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-credential-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.231958 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.231916 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istiod-ca-cert\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.233509 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.233485 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-podinfo\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.233785 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.233768 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-envoy\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.289289 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.289225 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-token\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.289422 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.289383 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mn58g\" (UniqueName: \"kubernetes.io/projected/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-kube-api-access-mn58g\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.338922 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.338882 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:33.765436 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.765401 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv"] Oct 30 18:59:33.805398 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:59:33.805371 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda394fd2c_8eb1_41e9_bd92_ca85d3bccbe3.slice/crio-d0d9c5840d0bcf7d97fff18781b48c6d5030ca0998ad72a88e69130a2ca390d3 WatchSource:0}: Error finding container d0d9c5840d0bcf7d97fff18781b48c6d5030ca0998ad72a88e69130a2ca390d3: Status 404 returned error can't find the container with id d0d9c5840d0bcf7d97fff18781b48c6d5030ca0998ad72a88e69130a2ca390d3 Oct 30 18:59:33.807859 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.807825 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:59:33.807950 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.807896 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:59:33.807950 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.807928 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:59:33.904923 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.904894 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:33.918717 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:33.918691 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:34.090971 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.090883 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:34.329562 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.329523 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq"] Oct 30 18:59:34.353693 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.352497 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.408145 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.408090 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-975cf5-openshift-default-dockercfg-g97ps\"" Oct 30 18:59:34.417123 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.417095 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq"] Oct 30 18:59:34.455157 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.453981 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43b1379e-42d3-4913-ab29-d766689a4a90-istio-podinfo\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.455157 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.454036 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-credential-socket\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.455157 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.454073 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43b1379e-42d3-4913-ab29-d766689a4a90-istio-token\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.455157 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.454151 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43b1379e-42d3-4913-ab29-d766689a4a90-istiod-ca-cert\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.455157 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.454178 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzlxz\" (UniqueName: \"kubernetes.io/projected/43b1379e-42d3-4913-ab29-d766689a4a90-kube-api-access-xzlxz\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.455157 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.454233 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-workload-socket\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.455157 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.454357 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-workload-certs\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.455157 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.454407 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-istio-data\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.455157 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.454460 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-istio-envoy\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.560615 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.555288 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43b1379e-42d3-4913-ab29-d766689a4a90-istiod-ca-cert\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.560615 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.555343 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzlxz\" (UniqueName: \"kubernetes.io/projected/43b1379e-42d3-4913-ab29-d766689a4a90-kube-api-access-xzlxz\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.560615 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.555381 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-workload-socket\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.560615 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.555474 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-workload-certs\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.560615 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.555516 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-istio-data\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.560615 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.555560 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-istio-envoy\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.560615 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.555654 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43b1379e-42d3-4913-ab29-d766689a4a90-istio-podinfo\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.560615 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.555694 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-credential-socket\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.560615 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.555728 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43b1379e-42d3-4913-ab29-d766689a4a90-istio-token\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.560615 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.559764 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43b1379e-42d3-4913-ab29-d766689a4a90-istiod-ca-cert\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.560615 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.560369 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-workload-socket\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.561324 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.560632 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-workload-certs\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.561324 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.560873 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-istio-data\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.562039 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.561924 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-credential-socket\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.563603 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.563543 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-istio-envoy\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.563859 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.563829 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43b1379e-42d3-4913-ab29-d766689a4a90-istio-podinfo\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.612341 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.612272 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43b1379e-42d3-4913-ab29-d766689a4a90-istio-token\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.615644 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.615616 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzlxz\" (UniqueName: \"kubernetes.io/projected/43b1379e-42d3-4913-ab29-d766689a4a90-kube-api-access-xzlxz\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.744274 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.744203 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:34.757012 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.756973 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:34.824677 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.824632 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" event={"ID":"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3","Type":"ContainerStarted","Data":"0eb55dd835c159c05b7055aa9a1a603889999cc57a1589f5ff4f7df9221f0653"} Oct 30 18:59:34.824908 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.824808 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" event={"ID":"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3","Type":"ContainerStarted","Data":"d0d9c5840d0bcf7d97fff18781b48c6d5030ca0998ad72a88e69130a2ca390d3"} Oct 30 18:59:34.866367 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.866287 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 18:59:34.954873 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:34.954083 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" podStartSLOduration=2.954063318 podStartE2EDuration="2.954063318s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:34.9532116 +0000 UTC m=+1245.443339424" watchObservedRunningTime="2026-10-30 18:59:34.954063318 +0000 UTC m=+1245.444191145" Oct 30 18:59:35.273336 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:35.273309 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq"] Oct 30 18:59:35.275211 ip-10-0-132-122 kubenswrapper[5894]: W1030 18:59:35.275172 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43b1379e_42d3_4913_ab29_d766689a4a90.slice/crio-9cb286d77315791d277506e77344cc2a32def491fb0ff80ee056e3f13402d969 WatchSource:0}: Error finding container 9cb286d77315791d277506e77344cc2a32def491fb0ff80ee056e3f13402d969: Status 404 returned error can't find the container with id 9cb286d77315791d277506e77344cc2a32def491fb0ff80ee056e3f13402d969 Oct 30 18:59:35.278026 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:35.277991 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:59:35.278105 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:35.278070 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:59:35.278143 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:35.278114 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 18:59:35.343704 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:35.343629 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:35.346025 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:35.345692 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 30 18:59:35.346025 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:35.345743 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" podUID="a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 30 18:59:35.847145 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:35.846482 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" event={"ID":"43b1379e-42d3-4913-ab29-d766689a4a90","Type":"ContainerStarted","Data":"469a6bab9d326996c01f9c9d782beebc8e837f38ff66edadf4f948d0931b05ba"} Oct 30 18:59:35.847145 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:35.846546 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" event={"ID":"43b1379e-42d3-4913-ab29-d766689a4a90","Type":"ContainerStarted","Data":"9cb286d77315791d277506e77344cc2a32def491fb0ff80ee056e3f13402d969"} Oct 30 18:59:36.347597 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:36.346996 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 30 18:59:36.347597 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:36.347060 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" podUID="a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 30 18:59:36.761609 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:36.761284 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:36.761609 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:36.761414 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 30 18:59:36.761609 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:36.761459 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" podUID="43b1379e-42d3-4913-ab29-d766689a4a90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 30 18:59:37.344599 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:37.342964 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 30 18:59:37.344599 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:37.343025 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" podUID="a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 30 18:59:37.761725 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:37.761688 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 30 18:59:37.762281 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:37.762251 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" podUID="43b1379e-42d3-4913-ab29-d766689a4a90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 30 18:59:38.343600 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:38.341708 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 30 18:59:38.343600 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:38.341775 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" podUID="a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 30 18:59:38.760598 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:38.759947 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 30 18:59:38.760598 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:38.760010 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" podUID="43b1379e-42d3-4913-ab29-d766689a4a90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 30 18:59:39.342598 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:39.340100 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 30 18:59:39.342598 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:39.340189 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" podUID="a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 30 18:59:39.674596 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:39.674134 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:39.779566 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:39.765702 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 30 18:59:39.779566 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:39.765828 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" podUID="43b1379e-42d3-4913-ab29-d766689a4a90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 30 18:59:40.342597 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:40.342037 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" start-of-body= Oct 30 18:59:40.342597 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:40.342120 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" podUID="a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.70:15021/healthz/ready\": dial tcp 10.132.0.70:15021: connect: connection refused" Oct 30 18:59:40.758342 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:40.758305 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 30 18:59:40.758562 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:40.758382 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" podUID="43b1379e-42d3-4913-ab29-d766689a4a90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 30 18:59:41.343913 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:41.343883 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:41.344374 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:41.344178 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:41.344855 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:41.344834 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 18:59:41.563355 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:41.563296 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" podStartSLOduration=7.563276821 podStartE2EDuration="7.563276821s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 18:59:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 18:59:35.97550566 +0000 UTC m=+1246.465633480" watchObservedRunningTime="2026-10-30 18:59:41.563276821 +0000 UTC m=+1252.053404642" Oct 30 18:59:41.762833 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:41.759415 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body= Oct 30 18:59:41.762833 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:41.759475 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" podUID="43b1379e-42d3-4913-ab29-d766689a4a90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" Oct 30 18:59:42.762629 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:42.762602 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:42.763026 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:42.763006 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:42.763800 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:42.763777 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 18:59:44.576976 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:44.576938 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:49.378366 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:49.378318 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:50.117085 ip-10-0-132-122 kubenswrapper[5894]: E1030 18:59:50.116998 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 18:59:50.639559 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:50.639526 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:54.071584 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:54.071550 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 18:59:58.764949 ip-10-0-132-122 kubenswrapper[5894]: I1030 18:59:58.764904 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:00:03.495341 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.495133 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:00:03.555941 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.555909 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl"] Oct 30 19:00:03.567688 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.567659 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.620311 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.620277 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl"] Oct 30 19:00:03.699706 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.699666 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-workload-certs\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.699880 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.699790 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-token\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.699880 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.699845 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-podinfo\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.699880 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.699864 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-credential-socket\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.700006 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.699883 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-envoy\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.700006 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.699958 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-workload-socket\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.700006 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.699993 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istiod-ca-cert\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.700153 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.700009 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7x49\" (UniqueName: \"kubernetes.io/projected/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-kube-api-access-s7x49\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.700208 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.700149 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-data\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.801713 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.801621 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-podinfo\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.801713 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.801671 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-credential-socket\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.801713 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.801700 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-envoy\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.801998 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.801732 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-workload-socket\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.801998 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.801770 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istiod-ca-cert\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.801998 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.801792 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7x49\" (UniqueName: \"kubernetes.io/projected/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-kube-api-access-s7x49\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.801998 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.801884 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-data\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.801998 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.801989 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-workload-certs\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.802237 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.802085 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-token\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.802237 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.802213 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-workload-socket\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.802348 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.802238 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-credential-socket\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.802542 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.802525 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-workload-certs\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.802632 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.802523 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-data\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.802670 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.802646 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istiod-ca-cert\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.804493 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.804472 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-envoy\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.804862 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.804838 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-podinfo\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.858968 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.858939 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7x49\" (UniqueName: \"kubernetes.io/projected/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-kube-api-access-s7x49\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.859174 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.859154 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-token\") pod \"e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:03.881176 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:03.881148 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:04.308602 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:04.308563 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl"] Oct 30 19:00:04.310297 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:00:04.310268 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14cda751_f1b9_4b48_9fe2_bcd0a2909d0c.slice/crio-613c89f370a3217d3dec1bb7c3528eee9e1bd0aac8bb018cc7affbe547291362 WatchSource:0}: Error finding container 613c89f370a3217d3dec1bb7c3528eee9e1bd0aac8bb018cc7affbe547291362: Status 404 returned error can't find the container with id 613c89f370a3217d3dec1bb7c3528eee9e1bd0aac8bb018cc7affbe547291362 Oct 30 19:00:04.312490 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:04.312462 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:04.312558 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:04.312525 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:04.312558 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:04.312553 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:04.870125 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:04.870090 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr"] Oct 30 19:00:04.873979 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:04.873960 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:04.933620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:04.933586 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr"] Oct 30 19:00:05.018717 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.018686 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-istio-data\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.018912 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.018723 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-credential-socket\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.018912 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.018837 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-workload-socket\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.018999 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.018959 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vg49g\" (UniqueName: \"kubernetes.io/projected/bcb8e536-b0a5-4517-8c98-a39793531709-kube-api-access-vg49g\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.019034 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.019018 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-workload-certs\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.019077 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.019042 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcb8e536-b0a5-4517-8c98-a39793531709-istio-token\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.019137 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.019088 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-istio-envoy\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.019181 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.019133 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcb8e536-b0a5-4517-8c98-a39793531709-istiod-ca-cert\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.019270 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.019253 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcb8e536-b0a5-4517-8c98-a39793531709-istio-podinfo\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.102230 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.102175 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" event={"ID":"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c","Type":"ContainerStarted","Data":"41a16f4cd21dccb62e094e2b7b86b0fe430f3ecec20b366ae4d9179ba5755340"} Oct 30 19:00:05.102562 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.102526 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" event={"ID":"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c","Type":"ContainerStarted","Data":"613c89f370a3217d3dec1bb7c3528eee9e1bd0aac8bb018cc7affbe547291362"} Oct 30 19:00:05.120690 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.120606 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-credential-socket\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.120690 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.120687 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-workload-socket\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.120876 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.120721 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vg49g\" (UniqueName: \"kubernetes.io/projected/bcb8e536-b0a5-4517-8c98-a39793531709-kube-api-access-vg49g\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.120876 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.120771 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-workload-certs\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.120876 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.120791 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcb8e536-b0a5-4517-8c98-a39793531709-istio-token\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.120876 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.120809 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-istio-envoy\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.120876 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.120825 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcb8e536-b0a5-4517-8c98-a39793531709-istiod-ca-cert\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.121115 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.120934 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcb8e536-b0a5-4517-8c98-a39793531709-istio-podinfo\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.121115 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.120958 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-credential-socket\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.121115 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.120981 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-istio-data\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.121115 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.121064 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-workload-socket\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.121332 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.121257 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-istio-data\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.121467 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.121447 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-workload-certs\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.121716 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.121694 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcb8e536-b0a5-4517-8c98-a39793531709-istiod-ca-cert\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.123220 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.123192 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-istio-envoy\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.123467 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.123449 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcb8e536-b0a5-4517-8c98-a39793531709-istio-podinfo\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.174661 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.167320 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh"] Oct 30 19:00:05.175454 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.175423 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.186039 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.182633 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vg49g\" (UniqueName: \"kubernetes.io/projected/bcb8e536-b0a5-4517-8c98-a39793531709-kube-api-access-vg49g\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.186039 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.185894 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcb8e536-b0a5-4517-8c98-a39793531709-istio-token\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.189475 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.189449 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:05.210694 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.210663 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh"] Oct 30 19:00:05.225623 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.225527 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" podStartSLOduration=2.225513545 podStartE2EDuration="2.225513545s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:05.224141385 +0000 UTC m=+1275.714269207" watchObservedRunningTime="2026-10-30 19:00:05.225513545 +0000 UTC m=+1275.715641415" Oct 30 19:00:05.355595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.342309 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-workload-certs\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.355595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.342409 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-envoy\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.355595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.342441 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-token\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.355595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.342529 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-podinfo\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.355595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.342560 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48ae9925-3d54-48a5-af7a-d6bc186ea365-istiod-ca-cert\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.355595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.342659 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2npqx\" (UniqueName: \"kubernetes.io/projected/48ae9925-3d54-48a5-af7a-d6bc186ea365-kube-api-access-2npqx\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.355595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.342703 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-workload-socket\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.355595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.342728 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-data\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.355595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.342799 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-credential-socket\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.445333 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.445288 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2npqx\" (UniqueName: \"kubernetes.io/projected/48ae9925-3d54-48a5-af7a-d6bc186ea365-kube-api-access-2npqx\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.446029 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.446008 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-workload-socket\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.446649 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.446631 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-data\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.447189 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.447173 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-credential-socket\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.447603 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.447587 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-workload-certs\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.448031 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.448017 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-envoy\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.448401 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.448385 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-token\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.448874 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.448855 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-podinfo\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.457784 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.446536 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-workload-socket\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.457784 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.447916 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-workload-certs\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.457784 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.447073 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-data\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.457784 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.457746 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48ae9925-3d54-48a5-af7a-d6bc186ea365-istiod-ca-cert\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.457784 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.447501 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-credential-socket\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.458633 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.458566 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48ae9925-3d54-48a5-af7a-d6bc186ea365-istiod-ca-cert\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.459998 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.459972 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-envoy\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.460352 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.460332 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-podinfo\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.502809 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.502778 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2npqx\" (UniqueName: \"kubernetes.io/projected/48ae9925-3d54-48a5-af7a-d6bc186ea365-kube-api-access-2npqx\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.503958 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.503935 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-token\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.557511 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.557238 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:05.677083 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.677053 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr"] Oct 30 19:00:05.684528 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.684499 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:05.684632 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.684558 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:05.684632 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.684599 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:05.881775 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.881739 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:05.883458 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.883432 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 30 19:00:05.883608 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.883483 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" podUID="14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 30 19:00:05.955143 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.955060 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh"] Oct 30 19:00:05.959092 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:00:05.959066 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48ae9925_3d54_48a5_af7a_d6bc186ea365.slice/crio-3e300ee09ac146c780c5b73b7d0dfdb01d57b18e503bc3208807580be70ab0d9 WatchSource:0}: Error finding container 3e300ee09ac146c780c5b73b7d0dfdb01d57b18e503bc3208807580be70ab0d9: Status 404 returned error can't find the container with id 3e300ee09ac146c780c5b73b7d0dfdb01d57b18e503bc3208807580be70ab0d9 Oct 30 19:00:05.961676 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.961645 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:05.961767 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.961735 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:05.961825 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:05.961765 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:06.126548 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:06.126505 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" event={"ID":"48ae9925-3d54-48a5-af7a-d6bc186ea365","Type":"ContainerStarted","Data":"b0a98eee3b434de1693b0eee3a355b5bad53d4cddcb4268e8f5687abff3fc7b7"} Oct 30 19:00:06.126763 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:06.126626 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" event={"ID":"48ae9925-3d54-48a5-af7a-d6bc186ea365","Type":"ContainerStarted","Data":"3e300ee09ac146c780c5b73b7d0dfdb01d57b18e503bc3208807580be70ab0d9"} Oct 30 19:00:06.159905 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:06.159865 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" event={"ID":"bcb8e536-b0a5-4517-8c98-a39793531709","Type":"ContainerStarted","Data":"ffe0b104ba9ba05a9a318fc35f18cefa7e5059cfafc4362b737fa14e467441f3"} Oct 30 19:00:06.160177 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:06.160152 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" event={"ID":"bcb8e536-b0a5-4517-8c98-a39793531709","Type":"ContainerStarted","Data":"0c96e0c747c5d1129926c4ff396d22fa410679622257b475b3df13e5e219cd8c"} Oct 30 19:00:06.191259 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:06.191233 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:06.192206 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:06.192188 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Oct 30 19:00:06.192343 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:06.192324 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" podUID="bcb8e536-b0a5-4517-8c98-a39793531709" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Oct 30 19:00:06.273977 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:06.273858 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" podStartSLOduration=1.273838117 podStartE2EDuration="1.273838117s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:06.272347617 +0000 UTC m=+1276.762475438" watchObservedRunningTime="2026-10-30 19:00:06.273838117 +0000 UTC m=+1276.763965938" Oct 30 19:00:06.391633 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:06.391557 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" podStartSLOduration=2.391544026 podStartE2EDuration="2.391544026s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:06.389612656 +0000 UTC m=+1276.879740477" watchObservedRunningTime="2026-10-30 19:00:06.391544026 +0000 UTC m=+1276.881671903" Oct 30 19:00:06.558341 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:06.558255 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:06.883641 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:06.883479 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 30 19:00:06.883641 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:06.883552 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" podUID="14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 30 19:00:07.195051 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:07.195007 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Oct 30 19:00:07.195310 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:07.195282 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" podUID="bcb8e536-b0a5-4517-8c98-a39793531709" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Oct 30 19:00:07.566864 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:07.566758 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 19:00:07.567157 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:07.567110 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" podUID="48ae9925-3d54-48a5-af7a-d6bc186ea365" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 19:00:07.882655 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:07.881967 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 30 19:00:07.882655 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:07.882027 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" podUID="14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 30 19:00:08.204601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:08.192959 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Oct 30 19:00:08.204601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:08.193020 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" podUID="bcb8e536-b0a5-4517-8c98-a39793531709" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Oct 30 19:00:08.455650 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:08.455482 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:00:08.565861 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:08.565828 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 19:00:08.566166 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:08.566115 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" podUID="48ae9925-3d54-48a5-af7a-d6bc186ea365" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 19:00:08.887060 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:08.886977 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 30 19:00:08.887327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:08.887301 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" podUID="14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 30 19:00:09.195026 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:09.194987 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Oct 30 19:00:09.195341 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:09.195309 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" podUID="bcb8e536-b0a5-4517-8c98-a39793531709" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Oct 30 19:00:09.562054 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:09.561959 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 19:00:09.562665 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:09.562631 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" podUID="48ae9925-3d54-48a5-af7a-d6bc186ea365" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 19:00:09.884653 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:09.883933 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 30 19:00:09.884653 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:09.883996 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" podUID="14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 30 19:00:10.212315 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:10.211888 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Oct 30 19:00:10.212315 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:10.211945 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" podUID="bcb8e536-b0a5-4517-8c98-a39793531709" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Oct 30 19:00:10.562030 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:10.561939 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 19:00:10.563238 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:10.563208 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" podUID="48ae9925-3d54-48a5-af7a-d6bc186ea365" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 19:00:10.919665 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:10.919606 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 30 19:00:10.920008 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:10.919959 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" podUID="14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 30 19:00:11.191203 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:11.191113 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Oct 30 19:00:11.191501 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:11.191472 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" podUID="bcb8e536-b0a5-4517-8c98-a39793531709" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Oct 30 19:00:11.579648 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:11.579184 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 19:00:11.579648 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:11.579242 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" podUID="48ae9925-3d54-48a5-af7a-d6bc186ea365" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 19:00:11.882283 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:11.882187 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" start-of-body= Oct 30 19:00:11.883258 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:11.883211 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" podUID="14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.72:15021/healthz/ready\": dial tcp 10.132.0.72:15021: connect: connection refused" Oct 30 19:00:12.206850 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:12.206726 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Oct 30 19:00:12.206850 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:12.206785 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" podUID="bcb8e536-b0a5-4517-8c98-a39793531709" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Oct 30 19:00:12.558309 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:12.558221 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 19:00:12.558604 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:12.558544 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" podUID="48ae9925-3d54-48a5-af7a-d6bc186ea365" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 19:00:13.022835 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:13.022801 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:13.023602 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:13.023568 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:13.025263 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:13.025239 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:00:13.191901 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:13.191862 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" start-of-body= Oct 30 19:00:13.192093 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:13.191941 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" podUID="bcb8e536-b0a5-4517-8c98-a39793531709" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": dial tcp 10.132.0.73:15021: connect: connection refused" Oct 30 19:00:13.558752 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:13.558711 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" start-of-body= Oct 30 19:00:13.558928 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:13.558782 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" podUID="48ae9925-3d54-48a5-af7a-d6bc186ea365" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.74:15021/healthz/ready\": dial tcp 10.132.0.74:15021: connect: connection refused" Oct 30 19:00:13.825658 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:13.817469 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:00:14.005559 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:14.005527 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:00:14.899174 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:14.899135 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:14.900704 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:14.900682 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:14.905125 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:14.905040 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:00:15.039944 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:15.039913 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:15.041162 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:15.041136 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:15.041977 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:15.041957 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:00:18.766938 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:18.766896 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:00:18.937191 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:18.937159 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:00:19.988155 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:19.988118 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x"] Oct 30 19:00:19.992631 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:19.992610 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.079614 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.079569 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-envoy\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.079754 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.079694 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-podinfo\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.079875 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.079855 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-workload-socket\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.079929 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.079910 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-credential-socket\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.079993 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.079976 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c7bl\" (UniqueName: \"kubernetes.io/projected/f247ccba-3633-4301-b8f2-41a0a8eab7ea-kube-api-access-4c7bl\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.080072 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.080050 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-workload-certs\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.080166 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.080150 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-token\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.080225 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.080203 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istiod-ca-cert\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.080300 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.080281 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-data\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.098360 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.098331 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x"] Oct 30 19:00:20.120656 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:00:20.120626 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:00:20.181994 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.181964 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-workload-socket\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.182246 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.182019 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-credential-socket\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.182246 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.182046 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c7bl\" (UniqueName: \"kubernetes.io/projected/f247ccba-3633-4301-b8f2-41a0a8eab7ea-kube-api-access-4c7bl\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.182246 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.182067 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-workload-certs\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.182246 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.182131 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-token\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.182246 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.182173 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istiod-ca-cert\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.182246 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.182202 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-data\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.182551 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.182286 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-envoy\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.182551 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.182326 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-podinfo\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.182551 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.182357 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-workload-socket\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.182551 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.182429 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-credential-socket\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.182727 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.182667 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-data\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.182785 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.182768 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-workload-certs\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.182874 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.182854 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istiod-ca-cert\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.184699 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.184677 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-envoy\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.184918 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.184899 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-podinfo\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.240830 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.240761 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-token\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.240977 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.240920 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c7bl\" (UniqueName: \"kubernetes.io/projected/f247ccba-3633-4301-b8f2-41a0a8eab7ea-kube-api-access-4c7bl\") pod \"e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.308491 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.308460 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:20.751836 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.751806 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x"] Oct 30 19:00:20.754147 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:00:20.754098 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf247ccba_3633_4301_b8f2_41a0a8eab7ea.slice/crio-c0bf8d3e7dfd8d51240f899347bf8afa8fbaebc605a4df391a0ff8fc902caf9e WatchSource:0}: Error finding container c0bf8d3e7dfd8d51240f899347bf8afa8fbaebc605a4df391a0ff8fc902caf9e: Status 404 returned error can't find the container with id c0bf8d3e7dfd8d51240f899347bf8afa8fbaebc605a4df391a0ff8fc902caf9e Oct 30 19:00:20.757120 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.757081 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:20.757225 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.757158 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:20.757225 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:20.757190 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:21.461029 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:21.460993 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" event={"ID":"f247ccba-3633-4301-b8f2-41a0a8eab7ea","Type":"ContainerStarted","Data":"1b7c620cead080673b81823bedc5f6f6f14586c9d04119027b35dd7f68655b82"} Oct 30 19:00:21.461475 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:21.461051 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" event={"ID":"f247ccba-3633-4301-b8f2-41a0a8eab7ea","Type":"ContainerStarted","Data":"c0bf8d3e7dfd8d51240f899347bf8afa8fbaebc605a4df391a0ff8fc902caf9e"} Oct 30 19:00:22.309119 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:22.309087 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:22.310700 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:22.310674 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Oct 30 19:00:22.310842 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:22.310737 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" podUID="f247ccba-3633-4301-b8f2-41a0a8eab7ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Oct 30 19:00:23.309388 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:23.309354 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Oct 30 19:00:23.309847 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:23.309407 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" podUID="f247ccba-3633-4301-b8f2-41a0a8eab7ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Oct 30 19:00:23.665229 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:23.665187 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:00:24.309279 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:24.309239 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Oct 30 19:00:24.309484 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:24.309295 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" podUID="f247ccba-3633-4301-b8f2-41a0a8eab7ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Oct 30 19:00:25.309509 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:25.309472 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Oct 30 19:00:25.309946 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:25.309522 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" podUID="f247ccba-3633-4301-b8f2-41a0a8eab7ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Oct 30 19:00:26.309116 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:26.309081 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Oct 30 19:00:26.309319 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:26.309133 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" podUID="f247ccba-3633-4301-b8f2-41a0a8eab7ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Oct 30 19:00:27.309426 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:27.309386 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body= Oct 30 19:00:27.309849 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:27.309452 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" podUID="f247ccba-3633-4301-b8f2-41a0a8eab7ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" Oct 30 19:00:27.499647 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:27.499615 5894 generic.go:356] "Generic (PLEG): container finished" podID="20536d00-75e4-4f08-acd3-02b64cd659c0" containerID="fd7aa2800175708e077e112aade52134e05250ebd6d79eca962a324e4f5c5ba7" exitCode=0 Oct 30 19:00:27.499819 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:27.499697 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" event={"ID":"20536d00-75e4-4f08-acd3-02b64cd659c0","Type":"ContainerDied","Data":"fd7aa2800175708e077e112aade52134e05250ebd6d79eca962a324e4f5c5ba7"} Oct 30 19:00:27.615283 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:27.615188 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" podStartSLOduration=8.615174903 podStartE2EDuration="8.615174903s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:21.653000234 +0000 UTC m=+1292.143128055" watchObservedRunningTime="2026-10-30 19:00:27.615174903 +0000 UTC m=+1298.105302729" Oct 30 19:00:28.314092 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:28.314062 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:28.314477 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:28.314430 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:28.315149 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:28.315128 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:00:28.378505 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:28.377098 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:00:28.678479 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:28.678447 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" Oct 30 19:00:28.799437 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:28.799407 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/20536d00-75e4-4f08-acd3-02b64cd659c0-archives-path\" (UniqueName: \"kubernetes.io/empty-dir/20536d00-75e4-4f08-acd3-02b64cd659c0-archives-path\") pod \"20536d00-75e4-4f08-acd3-02b64cd659c0\" (UID: \"20536d00-75e4-4f08-acd3-02b64cd659c0\") " Oct 30 19:00:28.799623 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:28.799492 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20536d00-75e4-4f08-acd3-02b64cd659c0-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20536d00-75e4-4f08-acd3-02b64cd659c0-trusted-ca-bundle\") pod \"20536d00-75e4-4f08-acd3-02b64cd659c0\" (UID: \"20536d00-75e4-4f08-acd3-02b64cd659c0\") " Oct 30 19:00:28.799698 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:28.799641 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/20536d00-75e4-4f08-acd3-02b64cd659c0-kube-api-access-zdz9q\" (UniqueName: \"kubernetes.io/projected/20536d00-75e4-4f08-acd3-02b64cd659c0-kube-api-access-zdz9q\") pod \"20536d00-75e4-4f08-acd3-02b64cd659c0\" (UID: \"20536d00-75e4-4f08-acd3-02b64cd659c0\") " Oct 30 19:00:28.799753 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:28.799709 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/20536d00-75e4-4f08-acd3-02b64cd659c0-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20536d00-75e4-4f08-acd3-02b64cd659c0-service-ca-bundle\") pod \"20536d00-75e4-4f08-acd3-02b64cd659c0\" (UID: \"20536d00-75e4-4f08-acd3-02b64cd659c0\") " Oct 30 19:00:28.799804 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:28.799788 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/20536d00-75e4-4f08-acd3-02b64cd659c0-archives-path" pod "20536d00-75e4-4f08-acd3-02b64cd659c0" (UID: "20536d00-75e4-4f08-acd3-02b64cd659c0"). InnerVolumeSpecName "archives-path". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:00:28.799941 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:28.799913 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20536d00-75e4-4f08-acd3-02b64cd659c0-trusted-ca-bundle" pod "20536d00-75e4-4f08-acd3-02b64cd659c0" (UID: "20536d00-75e4-4f08-acd3-02b64cd659c0"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:00:28.800248 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:28.800220 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20536d00-75e4-4f08-acd3-02b64cd659c0-service-ca-bundle" pod "20536d00-75e4-4f08-acd3-02b64cd659c0" (UID: "20536d00-75e4-4f08-acd3-02b64cd659c0"). InnerVolumeSpecName "service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:00:28.800248 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:28.800249 5894 reconciler_common.go:299] "Volume detached for volume \"archives-path\" (UniqueName: \"kubernetes.io/empty-dir/20536d00-75e4-4f08-acd3-02b64cd659c0-archives-path\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:00:28.800388 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:28.800264 5894 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20536d00-75e4-4f08-acd3-02b64cd659c0-trusted-ca-bundle\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:00:28.801779 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:28.801751 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20536d00-75e4-4f08-acd3-02b64cd659c0-kube-api-access-zdz9q" pod "20536d00-75e4-4f08-acd3-02b64cd659c0" (UID: "20536d00-75e4-4f08-acd3-02b64cd659c0"). InnerVolumeSpecName "kube-api-access-zdz9q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:00:28.901897 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:28.901812 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdz9q\" (UniqueName: \"kubernetes.io/projected/20536d00-75e4-4f08-acd3-02b64cd659c0-kube-api-access-zdz9q\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:00:28.901897 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:28.901847 5894 reconciler_common.go:299] "Volume detached for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/20536d00-75e4-4f08-acd3-02b64cd659c0-service-ca-bundle\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:00:29.513666 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:29.513639 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" Oct 30 19:00:29.514168 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:29.513676 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/periodic-gathering-cp4ws-pwgzh" event={"ID":"20536d00-75e4-4f08-acd3-02b64cd659c0","Type":"ContainerDied","Data":"70ea5cf7b087ffc9d0681f39b06d9f16ae71410e17af385dbf30d8de28494173"} Oct 30 19:00:29.514168 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:29.513719 5894 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70ea5cf7b087ffc9d0681f39b06d9f16ae71410e17af385dbf30d8de28494173" Oct 30 19:00:30.738042 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:30.738009 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:00:30.909512 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:30.909484 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:00:33.091224 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:33.091187 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:00:34.152599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.152277 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg"] Oct 30 19:00:34.172599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.169076 5894 cpu_manager.go:414] "containerMap: removing container" podUID="20536d00-75e4-4f08-acd3-02b64cd659c0" containerName="insights-gathering" Oct 30 19:00:34.172599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.169105 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="20536d00-75e4-4f08-acd3-02b64cd659c0" containerName="insights-gathering" Oct 30 19:00:34.172599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.169274 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="20536d00-75e4-4f08-acd3-02b64cd659c0" containerName="insights-gathering" Oct 30 19:00:34.176595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.174702 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.224426 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.224377 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-envoy\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.224844 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.224820 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-credential-socket\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.224982 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.224965 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-data\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.225108 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.225093 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-workload-certs\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.225236 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.225221 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-podinfo\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.225768 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.225745 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istiod-ca-cert\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.225919 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.225904 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-token\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.226040 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.226025 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-workload-socket\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.226146 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.226133 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvsmc\" (UniqueName: \"kubernetes.io/projected/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-kube-api-access-pvsmc\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.260032 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.259996 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg"] Oct 30 19:00:34.375596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.368848 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-credential-socket\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.375596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.368899 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-data\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.375596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.368936 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-workload-certs\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.375596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.368978 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-podinfo\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.375596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.369039 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istiod-ca-cert\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.375596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.369071 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-token\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.375596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.369104 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-workload-socket\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.375596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.369137 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvsmc\" (UniqueName: \"kubernetes.io/projected/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-kube-api-access-pvsmc\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.375596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.369257 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-envoy\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.423726 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.399311 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istiod-ca-cert\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.441206 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.424319 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-workload-socket\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.441206 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.424913 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-data\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.441206 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.425169 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-credential-socket\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.441206 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.425417 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-workload-certs\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.441206 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.427392 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:34.463258 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.459461 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-podinfo\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.465240 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.465184 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-envoy\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.710625 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.710511 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:34.737983 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.737911 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvsmc\" (UniqueName: \"kubernetes.io/projected/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-kube-api-access-pvsmc\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.742594 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.738624 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-token\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.841866 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.841829 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:34.844388 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.842162 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:34.876709 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.876663 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:34.906983 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:34.906260 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:35.171052 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:35.170999 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:35.448675 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:35.447870 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded" Oct 30 19:00:35.448675 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:35.448478 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 19:00:35.579783 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:35.579735 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:35.685841 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:35.685796 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:35.747653 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:35.722217 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:35.910235 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:35.910172 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:35.924495 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:35.922186 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded" Oct 30 19:00:35.999295 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:35.997188 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 19:00:35.999295 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:35.997324 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 19:00:35.999295 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:35.997369 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 19:00:35.999295 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:35.997403 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 19:00:35.999295 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:35.997424 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg"] Oct 30 19:00:36.018325 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:36.018282 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:36.018622 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:36.018534 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:36.018787 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:36.018766 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:36.099819 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:36.099790 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 19:00:36.116487 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:36.116469 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 19:00:36.175608 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:36.172393 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 19:00:36.785183 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:36.785144 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" event={"ID":"5a3f5720-0f19-4f6d-9609-1924e6d45b3e","Type":"ContainerStarted","Data":"fd5be8b365afbc1c2379b5192a44abe351aeae024d322abd470817ae5ba61023"} Oct 30 19:00:36.785384 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:36.785212 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" event={"ID":"5a3f5720-0f19-4f6d-9609-1924e6d45b3e","Type":"ContainerStarted","Data":"81c64e7ba1677b1e63ff319493661afbf72ca51d936063f22564dfd9f729ee81"} Oct 30 19:00:36.854598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:36.853835 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:36.916419 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:36.916359 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" podStartSLOduration=2.91634385 podStartE2EDuration="2.91634385s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:36.915516421 +0000 UTC m=+1307.405644240" watchObservedRunningTime="2026-10-30 19:00:36.91634385 +0000 UTC m=+1307.406471673" Oct 30 19:00:37.853489 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:37.853445 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 30 19:00:37.853917 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:37.853514 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" podUID="5a3f5720-0f19-4f6d-9609-1924e6d45b3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 30 19:00:38.799383 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:38.799348 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:00:38.849598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:38.847940 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 30 19:00:38.849598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:38.847998 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" podUID="5a3f5720-0f19-4f6d-9609-1924e6d45b3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 30 19:00:39.853419 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:39.852974 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 30 19:00:39.853419 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:39.853037 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" podUID="5a3f5720-0f19-4f6d-9609-1924e6d45b3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 30 19:00:40.848022 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:40.847984 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 30 19:00:40.848296 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:40.848269 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" podUID="5a3f5720-0f19-4f6d-9609-1924e6d45b3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 30 19:00:41.850596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:41.845930 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 30 19:00:41.850596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:41.845989 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" podUID="5a3f5720-0f19-4f6d-9609-1924e6d45b3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 30 19:00:42.843403 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:42.843372 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body= Oct 30 19:00:42.843613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:42.843426 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" podUID="5a3f5720-0f19-4f6d-9609-1924e6d45b3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" Oct 30 19:00:43.516997 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:43.516960 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:00:43.846853 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:43.846774 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:43.847257 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:43.847235 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:43.847834 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:43.847815 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:00:48.332054 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:48.332011 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:00:49.279120 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.279089 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69"] Oct 30 19:00:49.284331 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.284312 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.341982 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.341948 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69"] Oct 30 19:00:49.444396 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.444364 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-workload-certs\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.444535 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.444492 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/886729d6-2218-45ce-b082-64fe59f772a2-istio-podinfo\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.444535 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.444529 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/886729d6-2218-45ce-b082-64fe59f772a2-istio-token\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.444631 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.444552 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-istio-envoy\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.444667 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.444632 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-istio-data\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.444667 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.444653 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sps6v\" (UniqueName: \"kubernetes.io/projected/886729d6-2218-45ce-b082-64fe59f772a2-kube-api-access-sps6v\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.444746 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.444676 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/886729d6-2218-45ce-b082-64fe59f772a2-istiod-ca-cert\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.444834 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.444821 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-credential-socket\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.444868 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.444854 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-workload-socket\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.546375 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.546303 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/886729d6-2218-45ce-b082-64fe59f772a2-istio-podinfo\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.546375 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.546344 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/886729d6-2218-45ce-b082-64fe59f772a2-istio-token\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.546375 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.546364 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-istio-envoy\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.546646 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.546378 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-istio-data\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.546646 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.546394 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sps6v\" (UniqueName: \"kubernetes.io/projected/886729d6-2218-45ce-b082-64fe59f772a2-kube-api-access-sps6v\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.546646 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.546421 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/886729d6-2218-45ce-b082-64fe59f772a2-istiod-ca-cert\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.546646 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.546560 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-credential-socket\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.546646 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.546616 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-workload-socket\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.546908 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.546667 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-workload-certs\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.546968 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.546942 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-istio-data\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.547015 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.546972 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-credential-socket\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.547093 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.547068 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-workload-certs\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.547188 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.547169 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-workload-socket\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.547411 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.547383 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/886729d6-2218-45ce-b082-64fe59f772a2-istiod-ca-cert\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.548863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.548844 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-istio-envoy\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.548946 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.548885 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/886729d6-2218-45ce-b082-64fe59f772a2-istio-podinfo\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.603339 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.603304 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/886729d6-2218-45ce-b082-64fe59f772a2-istio-token\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.603456 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.603356 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sps6v\" (UniqueName: \"kubernetes.io/projected/886729d6-2218-45ce-b082-64fe59f772a2-kube-api-access-sps6v\") pod \"e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.898489 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.898451 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:49.982437 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.982405 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs"] Oct 30 19:00:49.987629 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:49.987604 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.053134 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.053104 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-workload-certs\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.053305 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.053144 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-workload-socket\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.053348 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.053302 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snz24\" (UniqueName: \"kubernetes.io/projected/11f90211-60b6-41d7-a44f-45198e9eb898-kube-api-access-snz24\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.053389 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.053347 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11f90211-60b6-41d7-a44f-45198e9eb898-istio-podinfo\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.053445 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.053426 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-credential-socket\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.053653 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.053635 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11f90211-60b6-41d7-a44f-45198e9eb898-istio-token\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.053699 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.053670 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11f90211-60b6-41d7-a44f-45198e9eb898-istiod-ca-cert\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.053742 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.053697 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-istio-envoy\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.053782 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.053755 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-istio-data\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.077205 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.077167 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs"] Oct 30 19:00:50.127526 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:00:50.127494 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:00:50.156028 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.155946 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11f90211-60b6-41d7-a44f-45198e9eb898-istio-token\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.156028 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.155994 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11f90211-60b6-41d7-a44f-45198e9eb898-istiod-ca-cert\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.156028 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.156013 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-istio-envoy\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.156231 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.156036 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-istio-data\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.156231 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.156079 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-workload-certs\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.156231 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.156105 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-workload-socket\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.156231 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.156172 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snz24\" (UniqueName: \"kubernetes.io/projected/11f90211-60b6-41d7-a44f-45198e9eb898-kube-api-access-snz24\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.156231 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.156204 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11f90211-60b6-41d7-a44f-45198e9eb898-istio-podinfo\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.156397 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.156233 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-credential-socket\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.156707 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.156681 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-workload-certs\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.156707 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.156722 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-credential-socket\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.156914 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.156892 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11f90211-60b6-41d7-a44f-45198e9eb898-istiod-ca-cert\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.157095 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.157076 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-workload-socket\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.157175 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.157153 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-istio-data\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.159271 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.159244 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-istio-envoy\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.159382 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.159326 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11f90211-60b6-41d7-a44f-45198e9eb898-istio-podinfo\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.221556 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.221522 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11f90211-60b6-41d7-a44f-45198e9eb898-istio-token\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.221759 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.221737 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-snz24\" (UniqueName: \"kubernetes.io/projected/11f90211-60b6-41d7-a44f-45198e9eb898-kube-api-access-snz24\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.305382 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.305348 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:50.330232 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.329925 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69"] Oct 30 19:00:50.332819 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:00:50.332522 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod886729d6_2218_45ce_b082_64fe59f772a2.slice/crio-490227e12be4afa0add2d2a2900cd0d09506b5b943508f608f2e5ffa907f6ba1 WatchSource:0}: Error finding container 490227e12be4afa0add2d2a2900cd0d09506b5b943508f608f2e5ffa907f6ba1: Status 404 returned error can't find the container with id 490227e12be4afa0add2d2a2900cd0d09506b5b943508f608f2e5ffa907f6ba1 Oct 30 19:00:50.335897 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.335866 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:50.336007 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.335936 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:50.336007 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.335975 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:50.510077 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.510035 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn"] Oct 30 19:00:50.514161 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.514141 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.578613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.576196 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn"] Oct 30 19:00:50.664323 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.664247 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-workload-certs\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.664323 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.664278 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a37b4ddc-1a56-4ebe-a548-d2681502698a-istiod-ca-cert\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.664323 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.664306 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-envoy\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.664565 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.664360 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-workload-socket\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.664565 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.664445 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-token\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.664565 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.664511 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-podinfo\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.664565 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.664552 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-data\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.664822 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.664725 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-credential-socket\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.664822 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.664763 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7nv8n\" (UniqueName: \"kubernetes.io/projected/a37b4ddc-1a56-4ebe-a548-d2681502698a-kube-api-access-7nv8n\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.695251 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.695222 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs"] Oct 30 19:00:50.696787 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:00:50.696757 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11f90211_60b6_41d7_a44f_45198e9eb898.slice/crio-69f3cd10340be3bc4d8722b16b0a630aa9bab872845235d2f18d6c3333031454 WatchSource:0}: Error finding container 69f3cd10340be3bc4d8722b16b0a630aa9bab872845235d2f18d6c3333031454: Status 404 returned error can't find the container with id 69f3cd10340be3bc4d8722b16b0a630aa9bab872845235d2f18d6c3333031454 Oct 30 19:00:50.702667 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.702635 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:50.702777 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.702699 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:50.702777 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.702725 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:50.766348 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.766322 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-workload-certs\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.766518 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.766356 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a37b4ddc-1a56-4ebe-a548-d2681502698a-istiod-ca-cert\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.766518 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.766399 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-envoy\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.766518 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.766426 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-workload-socket\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.766518 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.766498 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-token\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.766857 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.766623 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-podinfo\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.766857 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.766690 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-data\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.766857 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.766760 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-workload-certs\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.766857 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.766786 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-credential-socket\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.766857 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.766814 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7nv8n\" (UniqueName: \"kubernetes.io/projected/a37b4ddc-1a56-4ebe-a548-d2681502698a-kube-api-access-7nv8n\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.767103 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.767007 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-data\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.767151 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.767116 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a37b4ddc-1a56-4ebe-a548-d2681502698a-istiod-ca-cert\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.767355 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.767329 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-credential-socket\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.767510 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.767489 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-workload-socket\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.769538 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.769511 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-envoy\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.769873 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.769855 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-podinfo\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.823583 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.823552 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7nv8n\" (UniqueName: \"kubernetes.io/projected/a37b4ddc-1a56-4ebe-a548-d2681502698a-kube-api-access-7nv8n\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.823779 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.823548 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-token\") pod \"e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.829511 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.829480 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:50.870400 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.870348 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" event={"ID":"886729d6-2218-45ce-b082-64fe59f772a2","Type":"ContainerStarted","Data":"8b052e8b22a76b049786fe3e5a915cf20450d5ed3552371e8f4f82700e535bf9"} Oct 30 19:00:50.870841 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.870793 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" event={"ID":"886729d6-2218-45ce-b082-64fe59f772a2","Type":"ContainerStarted","Data":"490227e12be4afa0add2d2a2900cd0d09506b5b943508f608f2e5ffa907f6ba1"} Oct 30 19:00:50.878290 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.874534 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" event={"ID":"11f90211-60b6-41d7-a44f-45198e9eb898","Type":"ContainerStarted","Data":"a3021eab31d5bdaff70b125da4f2f54a46fe04d5694027883eb2ca42e689b4ac"} Oct 30 19:00:50.878290 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.874588 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" event={"ID":"11f90211-60b6-41d7-a44f-45198e9eb898","Type":"ContainerStarted","Data":"69f3cd10340be3bc4d8722b16b0a630aa9bab872845235d2f18d6c3333031454"} Oct 30 19:00:50.899332 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:50.899261 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:00:51.181013 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:51.180688 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" podStartSLOduration=2.180668174 podStartE2EDuration="2.180668174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:51.179141203 +0000 UTC m=+1321.669269064" watchObservedRunningTime="2026-10-30 19:00:51.180668174 +0000 UTC m=+1321.670795996" Oct 30 19:00:51.306861 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:51.305711 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:00:51.307358 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:51.307325 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 19:00:51.307496 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:51.307382 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" podUID="11f90211-60b6-41d7-a44f-45198e9eb898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 19:00:51.308775 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:51.308645 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" podStartSLOduration=2.308627854 podStartE2EDuration="2.308627854s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:51.307843506 +0000 UTC m=+1321.797971331" watchObservedRunningTime="2026-10-30 19:00:51.308627854 +0000 UTC m=+1321.798755676" Oct 30 19:00:51.462237 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:51.462125 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn"] Oct 30 19:00:51.543679 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:00:51.543646 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda37b4ddc_1a56_4ebe_a548_d2681502698a.slice/crio-3e123210d6e2de690cb676edf42d109a178da02c332d5db50e0e49f7c749be9a WatchSource:0}: Error finding container 3e123210d6e2de690cb676edf42d109a178da02c332d5db50e0e49f7c749be9a: Status 404 returned error can't find the container with id 3e123210d6e2de690cb676edf42d109a178da02c332d5db50e0e49f7c749be9a Oct 30 19:00:51.581471 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:51.576820 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:51.581471 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:51.576936 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:51.581471 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:51.576982 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:00:51.970653 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:51.968853 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 30 19:00:51.970653 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:51.968910 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" podUID="886729d6-2218-45ce-b082-64fe59f772a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 30 19:00:52.080665 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:52.043097 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" event={"ID":"a37b4ddc-1a56-4ebe-a548-d2681502698a","Type":"ContainerStarted","Data":"3e123210d6e2de690cb676edf42d109a178da02c332d5db50e0e49f7c749be9a"} Oct 30 19:00:52.308635 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:52.307052 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 19:00:52.308635 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:52.307107 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" podUID="11f90211-60b6-41d7-a44f-45198e9eb898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 19:00:52.758618 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:52.757752 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:52.911487 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:52.911451 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 30 19:00:52.938950 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:52.938810 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" podUID="886729d6-2218-45ce-b082-64fe59f772a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 30 19:00:53.100048 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:53.099955 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" event={"ID":"a37b4ddc-1a56-4ebe-a548-d2681502698a","Type":"ContainerStarted","Data":"c1be93fb4c492a5ee08bf28e7bfe36420a1f448b7c3bf5ae59c5fd27513f3b19"} Oct 30 19:00:53.125599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:53.112749 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:53.125937 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:53.125899 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:53.275605 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:53.223539 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" podStartSLOduration=3.223522473 podStartE2EDuration="3.223522473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:00:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:00:53.217840069 +0000 UTC m=+1323.707967890" watchObservedRunningTime="2026-10-30 19:00:53.223522473 +0000 UTC m=+1323.713650297" Oct 30 19:00:53.316602 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:53.313144 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 19:00:53.316602 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:53.313221 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" podUID="11f90211-60b6-41d7-a44f-45198e9eb898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 19:00:53.469551 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:53.469513 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:00:53.725651 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:53.718667 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded" Oct 30 19:00:53.765599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:53.764652 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:53.765599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:53.765125 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 19:00:53.831268 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:53.831233 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:00:53.831627 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:53.831603 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 30 19:00:53.831800 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:53.831775 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" podUID="a37b4ddc-1a56-4ebe-a548-d2681502698a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 30 19:00:53.947659 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:53.941277 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 30 19:00:53.947965 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:53.947916 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" podUID="886729d6-2218-45ce-b082-64fe59f772a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 30 19:00:54.129882 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:54.129786 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:54.130539 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:54.130513 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 19:00:54.145598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:54.132367 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:54.147042 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:54.147012 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 19:00:54.157266 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:54.156479 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:54.188600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:54.168656 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:54.190821 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:54.190781 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:54.308989 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:54.308951 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 19:00:54.328784 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:54.317546 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" podUID="11f90211-60b6-41d7-a44f-45198e9eb898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 19:00:54.434339 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:54.434296 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 19:00:54.570138 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:54.570106 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 19:00:54.880903 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:54.880867 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 30 19:00:54.881527 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:54.881494 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" podUID="a37b4ddc-1a56-4ebe-a548-d2681502698a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 30 19:00:54.909607 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:54.905214 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 30 19:00:54.909896 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:54.909860 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" podUID="886729d6-2218-45ce-b082-64fe59f772a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 30 19:00:55.154654 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:55.146262 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded" Oct 30 19:00:55.190600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:55.181842 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 19:00:55.215599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:55.207189 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:00:55.215599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:55.207531 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 19:00:55.384399 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:55.383639 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 19:00:55.419365 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:55.395549 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 19:00:55.419365 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:55.395636 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" podUID="11f90211-60b6-41d7-a44f-45198e9eb898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 19:00:55.840652 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:55.838845 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/health\": context deadline exceeded" Oct 30 19:00:55.908160 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:55.908123 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 30 19:00:56.015787 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:56.015735 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" podUID="886729d6-2218-45ce-b082-64fe59f772a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 30 19:00:56.033807 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:56.010422 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 30 19:00:56.034100 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:56.034069 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" podUID="a37b4ddc-1a56-4ebe-a548-d2681502698a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 30 19:00:56.324995 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:56.324926 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 19:00:56.337164 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:56.337122 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" podUID="11f90211-60b6-41d7-a44f-45198e9eb898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 19:00:56.861004 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:56.860966 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 30 19:00:56.864704 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:56.864666 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" podUID="a37b4ddc-1a56-4ebe-a548-d2681502698a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 30 19:00:56.916601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:56.904250 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 30 19:00:56.916601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:56.908010 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" podUID="886729d6-2218-45ce-b082-64fe59f772a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 30 19:00:57.317400 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:57.317349 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 19:00:57.317644 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:57.317415 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" podUID="11f90211-60b6-41d7-a44f-45198e9eb898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 19:00:57.834900 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:57.834865 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 30 19:00:57.838736 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:57.838650 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" podUID="a37b4ddc-1a56-4ebe-a548-d2681502698a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 30 19:00:57.905115 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:57.905081 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 30 19:00:57.905425 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:57.905377 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" podUID="886729d6-2218-45ce-b082-64fe59f772a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 30 19:00:58.278704 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:58.278647 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:00:58.308530 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:58.308455 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 19:00:58.309357 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:58.308907 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" podUID="11f90211-60b6-41d7-a44f-45198e9eb898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 19:00:58.831422 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:58.830949 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 30 19:00:58.831422 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:58.831011 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" podUID="a37b4ddc-1a56-4ebe-a548-d2681502698a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 30 19:00:58.904029 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:58.903990 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 30 19:00:58.904340 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:58.904275 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" podUID="886729d6-2218-45ce-b082-64fe59f772a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 30 19:00:59.306354 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:59.306315 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 19:00:59.306823 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:59.306372 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" podUID="11f90211-60b6-41d7-a44f-45198e9eb898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 19:00:59.506666 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:59.506605 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:00:59.831021 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:59.830987 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 30 19:00:59.831322 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:59.831272 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" podUID="a37b4ddc-1a56-4ebe-a548-d2681502698a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 30 19:00:59.899493 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:59.899449 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" start-of-body= Oct 30 19:00:59.899671 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:00:59.899524 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" podUID="886729d6-2218-45ce-b082-64fe59f772a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": dial tcp 10.132.0.77:15021: connect: connection refused" Oct 30 19:01:00.306657 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:00.306618 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" start-of-body= Oct 30 19:01:00.307047 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:00.306681 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" podUID="11f90211-60b6-41d7-a44f-45198e9eb898" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.78:15021/healthz/ready\": dial tcp 10.132.0.78:15021: connect: connection refused" Oct 30 19:01:00.829999 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:00.829958 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body= Oct 30 19:01:00.830201 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:00.830028 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" podUID="a37b4ddc-1a56-4ebe-a548-d2681502698a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" Oct 30 19:01:01.475972 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:01.475940 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:01:01.477791 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:01.477446 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:01:01.478351 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:01.478272 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:01:01.597109 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:01.597076 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:01:01.972231 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:01.967022 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:01:01.972231 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:01.967111 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" podUID="886729d6-2218-45ce-b082-64fe59f772a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:02.621197 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:02.621155 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:01:02.621689 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:02.621669 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:01:02.622840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:02.622817 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:01:02.967629 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:02.967553 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:01:02.967820 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:02.967670 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" podUID="886729d6-2218-45ce-b082-64fe59f772a2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.77:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:03.032595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:03.032550 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:01:03.033962 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:03.033936 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:01:03.034543 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:03.034522 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:01:04.607857 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:04.607814 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:01:09.303005 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:09.302968 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:01:14.448600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:14.444706 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:14.659555 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:14.659514 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:01:14.973719 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:14.973673 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:15.163798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:15.163755 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded" Oct 30 19:01:15.168998 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:15.168644 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:15.191413 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:15.190653 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:15.389774 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:15.389675 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:15.447978 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:15.447921 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:15.450086 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:15.449054 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 19:01:15.655657 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:15.581861 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded" Oct 30 19:01:15.655657 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:15.622675 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:15.667776 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:15.667679 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:15.709971 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:15.707649 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:15.925015 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:15.924666 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/health\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:15.967598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:15.956691 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:01:15.967598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:15.956799 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" podUID="bcb8e536-b0a5-4517-8c98-a39793531709" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.73:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:16.182657 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:16.172128 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:16.182657 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:16.174722 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 19:01:16.182657 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:16.175842 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:16.194608 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:16.186721 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 19:01:16.215614 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:16.207289 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:16.215614 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:16.208343 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 19:01:16.285768 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:16.285723 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": context deadline exceeded" Oct 30 19:01:16.477663 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:16.475302 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded" Oct 30 19:01:16.522565 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:16.522493 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 19:01:16.582802 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:16.582758 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded" Oct 30 19:01:16.584887 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:16.584861 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 19:01:16.846642 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:16.833377 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:16.846642 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:16.838754 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 19:01:17.240615 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:17.227802 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:17.264230 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:17.264187 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:17.517684 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:17.479943 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:17.600863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:17.600819 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:17.844679 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:17.841730 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:18.134655 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:18.134349 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 19:01:18.172917 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:18.172889 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 19:01:18.199271 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:18.199242 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 19:01:18.318599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:18.315550 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:18.362029 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:18.361679 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:18.414654 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:18.409385 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 19:01:18.807665 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:18.803647 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 19:01:19.274500 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:19.274471 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:01:19.461412 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:19.461368 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:01:20.123788 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:01:20.123734 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:01:20.392549 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:20.391545 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:01:22.154535 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.154505 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2"] Oct 30 19:01:22.159433 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.159411 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:22.210975 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.210947 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b42c4711-cc79-4024-b2b9-c6294004c373-tls-certs\") pod \"distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:22.211160 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.210993 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7zkl\" (UniqueName: \"kubernetes.io/projected/b42c4711-cc79-4024-b2b9-c6294004c373-kube-api-access-h7zkl\") pod \"distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:22.211236 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.211158 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-home\") pod \"distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:22.211299 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.211231 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-tmp-dir\") pod \"distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:22.211299 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.211270 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-dshm\") pod \"distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:22.211299 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.211285 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-model-cache\") pod \"distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:22.216240 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.216211 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2"] Oct 30 19:01:22.262475 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.262449 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-1290a6\"/\"distinct-2-w4-1290a6-kserve-self-signed-certs\"" Oct 30 19:01:22.289113 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.289088 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:01:22.315546 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.314197 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-home\") pod \"distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:22.316319 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.316291 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-tmp-dir\") pod \"distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:22.317258 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.317234 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-dshm\") pod \"distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:22.317500 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.317480 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-model-cache\") pod \"distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:22.317854 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.317835 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b42c4711-cc79-4024-b2b9-c6294004c373-tls-certs\") pod \"distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:22.318082 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.318063 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7zkl\" (UniqueName: \"kubernetes.io/projected/b42c4711-cc79-4024-b2b9-c6294004c373-kube-api-access-h7zkl\") pod \"distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:22.319541 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.319521 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-home\") pod \"distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:22.320114 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.320094 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-tmp-dir\") pod \"distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:22.328794 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.321885 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-model-cache\") pod \"distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:22.346862 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.346820 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b42c4711-cc79-4024-b2b9-c6294004c373-tls-certs\") pod \"distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:22.376600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.376141 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-dshm\") pod \"distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:22.403601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.399405 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7zkl\" (UniqueName: \"kubernetes.io/projected/b42c4711-cc79-4024-b2b9-c6294004c373-kube-api-access-h7zkl\") pod \"distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:22.482238 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:22.480950 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:23.094492 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:23.094464 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2"] Oct 30 19:01:23.096252 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:01:23.096228 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb42c4711_cc79_4024_b2b9_c6294004c373.slice/crio-fc25bffcfdb5fea4370b957147e5418448b18491a0fb23c8339c8476a5acb9df WatchSource:0}: Error finding container fc25bffcfdb5fea4370b957147e5418448b18491a0fb23c8339c8476a5acb9df: Status 404 returned error can't find the container with id fc25bffcfdb5fea4370b957147e5418448b18491a0fb23c8339c8476a5acb9df Oct 30 19:01:23.968191 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:23.968158 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" event={"ID":"b42c4711-cc79-4024-b2b9-c6294004c373","Type":"ContainerStarted","Data":"7fb7221e4aae820f53bffa99bd0af35cc22ac02e91de796e862eb9462193ea2f"} Oct 30 19:01:23.968615 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:23.968203 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" event={"ID":"b42c4711-cc79-4024-b2b9-c6294004c373","Type":"ContainerStarted","Data":"fc25bffcfdb5fea4370b957147e5418448b18491a0fb23c8339c8476a5acb9df"} Oct 30 19:01:24.090187 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:24.090137 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podStartSLOduration=2.090123148 podStartE2EDuration="2.090123148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:24.089183232 +0000 UTC m=+1354.579311053" watchObservedRunningTime="2026-10-30 19:01:24.090123148 +0000 UTC m=+1354.580250968" Oct 30 19:01:24.481690 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:24.481662 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:25.402624 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:25.402540 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:01:30.068628 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:30.068567 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:01:32.482253 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:32.482171 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:32.496303 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:32.496278 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:33.034595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:33.034551 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:34.991550 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:34.991517 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:01:35.122686 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.122631 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57"] Oct 30 19:01:35.130940 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.130908 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.174220 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.174192 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57"] Oct 30 19:01:35.214152 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.214118 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb0794ea-65d0-4050-81eb-2b961d482d33-istiod-ca-cert\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.214331 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.214168 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-workload-certs\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.214331 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.214296 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-credential-socket\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.214464 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.214366 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-podinfo\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.214464 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.214416 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-data\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.214597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.214504 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-envoy\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.214597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.214529 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-workload-socket\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.214597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.214557 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnt6m\" (UniqueName: \"kubernetes.io/projected/bb0794ea-65d0-4050-81eb-2b961d482d33-kube-api-access-dnt6m\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.214768 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.214638 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-token\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.316026 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.315951 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb0794ea-65d0-4050-81eb-2b961d482d33-istiod-ca-cert\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.316026 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.315992 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-workload-certs\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.316214 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.316047 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-credential-socket\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.316214 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.316078 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-podinfo\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.316214 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.316119 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-data\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.316214 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.316162 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-envoy\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.316214 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.316198 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-workload-socket\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.316440 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.316281 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dnt6m\" (UniqueName: \"kubernetes.io/projected/bb0794ea-65d0-4050-81eb-2b961d482d33-kube-api-access-dnt6m\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.316440 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.316339 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-token\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.316726 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.316696 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-workload-certs\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.316867 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.316778 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-credential-socket\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.316867 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.316807 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-workload-socket\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.316867 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.316845 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-data\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.316996 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.316882 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb0794ea-65d0-4050-81eb-2b961d482d33-istiod-ca-cert\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.318544 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.318523 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-envoy\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.318796 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.318776 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-podinfo\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.373773 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.373744 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-token\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.374153 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.374133 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnt6m\" (UniqueName: \"kubernetes.io/projected/bb0794ea-65d0-4050-81eb-2b961d482d33-kube-api-access-dnt6m\") pod \"e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.448588 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.448545 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:35.881632 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.881609 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57"] Oct 30 19:01:35.883347 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:01:35.883318 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb0794ea_65d0_4050_81eb_2b961d482d33.slice/crio-d301c62d4335105e16c7f2c5500bc4891a537d757214b09d35823a2759f9eba6 WatchSource:0}: Error finding container d301c62d4335105e16c7f2c5500bc4891a537d757214b09d35823a2759f9eba6: Status 404 returned error can't find the container with id d301c62d4335105e16c7f2c5500bc4891a537d757214b09d35823a2759f9eba6 Oct 30 19:01:35.885815 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.885788 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:01:35.885899 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.885845 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:01:35.885899 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:35.885872 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:01:36.055631 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:36.055597 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" event={"ID":"bb0794ea-65d0-4050-81eb-2b961d482d33","Type":"ContainerStarted","Data":"cf1a378a78e8889c87103659cafbc0a2c3bd99865dccd9c6261398862f9d2631"} Oct 30 19:01:36.056032 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:36.055654 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" event={"ID":"bb0794ea-65d0-4050-81eb-2b961d482d33","Type":"ContainerStarted","Data":"d301c62d4335105e16c7f2c5500bc4891a537d757214b09d35823a2759f9eba6"} Oct 30 19:01:36.176233 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:36.176173 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" podStartSLOduration=1.176157307 podStartE2EDuration="1.176157307s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:01:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:01:36.174808138 +0000 UTC m=+1366.664935988" watchObservedRunningTime="2026-10-30 19:01:36.176157307 +0000 UTC m=+1366.666285126" Oct 30 19:01:36.451651 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:36.451404 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:36.453401 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:36.453371 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 30 19:01:36.453561 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:36.453473 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" podUID="bb0794ea-65d0-4050-81eb-2b961d482d33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 30 19:01:37.448977 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:37.448937 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 30 19:01:37.449565 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:37.449531 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" podUID="bb0794ea-65d0-4050-81eb-2b961d482d33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 30 19:01:38.449954 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:38.449915 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 30 19:01:38.450601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:38.450548 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" podUID="bb0794ea-65d0-4050-81eb-2b961d482d33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 30 19:01:39.450250 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:39.450216 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 30 19:01:39.450869 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:39.450837 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" podUID="bb0794ea-65d0-4050-81eb-2b961d482d33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 30 19:01:39.802660 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:39.792634 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:01:40.448966 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:40.448930 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 30 19:01:40.449259 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:40.449233 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" podUID="bb0794ea-65d0-4050-81eb-2b961d482d33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 30 19:01:41.451959 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:41.451913 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 30 19:01:41.452426 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:41.451976 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" podUID="bb0794ea-65d0-4050-81eb-2b961d482d33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 30 19:01:42.449413 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:42.449376 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" start-of-body= Oct 30 19:01:42.449599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:42.449425 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" podUID="bb0794ea-65d0-4050-81eb-2b961d482d33" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.81:15021/healthz/ready\": dial tcp 10.132.0.81:15021: connect: connection refused" Oct 30 19:01:43.453256 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:43.453225 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:43.453756 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:43.453690 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:43.454496 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:43.454475 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:01:44.673230 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:44.673199 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:01:48.389731 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:48.389679 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:48.523717 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:48.523657 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded" Oct 30 19:01:48.526719 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:48.526682 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:48.706720 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:48.706670 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:48.898726 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:48.898677 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:49.043655 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:49.034665 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:49.266599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:49.266166 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded" Oct 30 19:01:49.394597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:49.390797 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded" Oct 30 19:01:49.394597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:49.391258 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 19:01:49.527711 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:49.527661 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:49.528250 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:49.527664 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:49.529703 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:49.528894 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 19:01:49.529913 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:49.529898 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 19:01:49.960600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:49.955654 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded" Oct 30 19:01:50.290091 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:50.289982 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:50.379597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:50.323944 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded" Oct 30 19:01:50.522604 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:50.486000 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:50.522604 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:50.504019 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded" Oct 30 19:01:50.663982 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:50.663925 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded" Oct 30 19:01:50.821598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:50.820768 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:51.011651 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:51.011089 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:51.058926 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:51.058876 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:51.356654 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:51.335643 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:51.396735 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:01:51.396686 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:01:51.502505 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:51.502450 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded" Oct 30 19:01:51.503517 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:01:51.503487 5894 kubelet.go:2711] "Housekeeping took longer than expected" err="housekeeping took too long" expected="1s" actual="1.857s" Oct 30 19:01:51.503840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:51.503822 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 19:01:51.504012 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:51.503998 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 19:01:51.504168 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:51.504155 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 19:01:51.504326 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:51.504313 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:51.546481 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:51.513796 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:51.582046 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:51.520462 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:51.582420 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:51.582386 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 19:01:51.671175 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:51.671135 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded" Oct 30 19:01:51.932659 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:51.929198 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:51.981951 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:51.981905 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:01:52.027697 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:52.026977 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:52.131600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:52.127675 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:52.396655 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:52.352096 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded" Oct 30 19:01:52.436598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:52.406032 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:52.504933 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:52.504891 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:52.683654 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:52.683062 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:52.760736 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:52.721885 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:52.760736 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:52.724131 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded" Oct 30 19:01:52.942586 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:52.939554 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:53.033391 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:53.033342 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:53.155693 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:53.155646 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:53.352483 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:53.352390 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:53.502996 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:53.502678 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:53.507470 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:53.506656 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:53.685811 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:53.684667 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:53.723749 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:53.723700 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:53.724851 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:53.724819 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:01:53.884425 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:53.884393 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 19:01:53.964255 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:53.964167 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 19:01:53.991737 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:53.991705 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 19:01:54.042091 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:54.042041 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded" Oct 30 19:01:54.279631 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:54.274423 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 19:01:54.280102 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:54.280081 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 19:01:54.304441 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:54.292785 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 19:01:54.316844 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:54.312208 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:01:54.334191 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:54.334167 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 19:01:57.785321 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:01:57.785284 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:02.622727 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:02.622662 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:04.308728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.308688 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq"] Oct 30 19:02:04.314269 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.314243 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.362617 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.362569 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254"] Oct 30 19:02:04.366672 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.366649 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.414127 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.414095 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq"] Oct 30 19:02:04.426427 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.426390 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-data\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.426612 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.426445 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-workload-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.426612 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.426555 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-podinfo\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.426747 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.426637 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bgwn\" (UniqueName: \"kubernetes.io/projected/43bf10dc-7107-4803-a386-c98e0a9c3bc2-kube-api-access-8bgwn\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.426747 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.426739 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-credential-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.426852 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.426781 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-workload-certs\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.426936 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.426912 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istiod-ca-cert\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.426995 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.426957 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-envoy\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.427091 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.427073 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-token\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.462612 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.462563 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254"] Oct 30 19:02:04.528667 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.528636 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-data\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.528843 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.528712 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istiod-ca-cert\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.528843 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.528773 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-credential-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.528843 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.528801 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-workload-certs\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.528843 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.528824 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-envoy\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.529021 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.528868 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-token\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.529021 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.529007 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-workload-certs\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.529127 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.529106 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-podinfo\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.529182 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.529143 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istiod-ca-cert\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.529235 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.529182 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-credential-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.529235 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.529198 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-envoy\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.529235 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.529221 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-workload-certs\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.529391 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.529302 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-token\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.529391 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.529345 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-data\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.529391 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.529365 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bqtmq\" (UniqueName: \"kubernetes.io/projected/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-kube-api-access-bqtmq\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.529557 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.529396 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-workload-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.529557 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.529443 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-workload-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.529713 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.529591 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-podinfo\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.529713 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.529639 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8bgwn\" (UniqueName: \"kubernetes.io/projected/43bf10dc-7107-4803-a386-c98e0a9c3bc2-kube-api-access-8bgwn\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.529713 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.529667 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-credential-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.529713 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.529677 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-data\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.529930 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.529880 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-workload-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.529992 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.529940 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istiod-ca-cert\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.531470 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.531454 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-envoy\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.531899 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.531883 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-podinfo\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.585702 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.585628 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-token\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.585954 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.585933 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bgwn\" (UniqueName: \"kubernetes.io/projected/43bf10dc-7107-4803-a386-c98e0a9c3bc2-kube-api-access-8bgwn\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.628543 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.628516 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:04.631066 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.631035 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-workload-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.631202 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.631081 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-credential-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.631202 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.631116 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-data\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.631202 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.631198 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istiod-ca-cert\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.631370 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.631227 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-workload-certs\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.631370 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.631259 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-envoy\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.631370 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.631299 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-token\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.631514 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.631397 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-podinfo\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.631514 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.631470 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-workload-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.631629 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.631557 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bqtmq\" (UniqueName: \"kubernetes.io/projected/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-kube-api-access-bqtmq\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.631691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.631618 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-data\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.631691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.631632 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-workload-certs\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.631783 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.631699 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-credential-socket\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.632091 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.632066 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istiod-ca-cert\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.633553 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.633526 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-envoy\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.634467 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.634446 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-podinfo\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.688671 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.688642 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-token\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.689252 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.689231 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bqtmq\" (UniqueName: \"kubernetes.io/projected/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-kube-api-access-bqtmq\") pod \"e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:04.980121 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:04.980091 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:05.072042 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:02:05.071996 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43bf10dc_7107_4803_a386_c98e0a9c3bc2.slice/crio-8933cddcdd22c429a0fca6afc4df64bb2c5220f05cc6ed8401ad28641899a548 WatchSource:0}: Error finding container 8933cddcdd22c429a0fca6afc4df64bb2c5220f05cc6ed8401ad28641899a548: Status 404 returned error can't find the container with id 8933cddcdd22c429a0fca6afc4df64bb2c5220f05cc6ed8401ad28641899a548 Oct 30 19:02:05.072162 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:05.072139 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq"] Oct 30 19:02:05.074680 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:05.074640 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:02:05.074799 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:05.074735 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:02:05.074799 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:05.074778 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:02:05.350134 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:05.350054 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" event={"ID":"43bf10dc-7107-4803-a386-c98e0a9c3bc2","Type":"ContainerStarted","Data":"2fd54502511abc944d1a825d40a66d16f9cc22faf9c1a3f4e9cb6070e5a94adc"} Oct 30 19:02:05.350134 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:05.350100 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" event={"ID":"43bf10dc-7107-4803-a386-c98e0a9c3bc2","Type":"ContainerStarted","Data":"8933cddcdd22c429a0fca6afc4df64bb2c5220f05cc6ed8401ad28641899a548"} Oct 30 19:02:05.360856 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:05.360829 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254"] Oct 30 19:02:05.361229 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:02:05.361175 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2b2e6c8_764f_4f40_9973_ebd7d7b1cfdb.slice/crio-c766f69c6c0942fc293744b7fcae9549841e93e560f096fbdd9e71887a966e12 WatchSource:0}: Error finding container c766f69c6c0942fc293744b7fcae9549841e93e560f096fbdd9e71887a966e12: Status 404 returned error can't find the container with id c766f69c6c0942fc293744b7fcae9549841e93e560f096fbdd9e71887a966e12 Oct 30 19:02:05.363548 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:05.363523 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:02:05.363640 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:05.363605 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:02:05.363695 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:05.363650 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:02:05.467779 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:05.467721 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" podStartSLOduration=1.4677012249999999 podStartE2EDuration="1.467701225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:05.465600933 +0000 UTC m=+1395.955728755" watchObservedRunningTime="2026-10-30 19:02:05.467701225 +0000 UTC m=+1395.957829049" Oct 30 19:02:05.628704 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:05.628620 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:06.383930 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:06.383881 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" event={"ID":"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb","Type":"ContainerStarted","Data":"3a1ec1750c46d2c6084e84213c21ee7084d45aab104ce63050bb5f7e99ea6adb"} Oct 30 19:02:06.384436 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:06.383960 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" event={"ID":"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb","Type":"ContainerStarted","Data":"c766f69c6c0942fc293744b7fcae9549841e93e560f096fbdd9e71887a966e12"} Oct 30 19:02:06.611344 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:06.611298 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" podStartSLOduration=2.611283871 podStartE2EDuration="2.611283871s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:06.608867138 +0000 UTC m=+1397.098994959" watchObservedRunningTime="2026-10-30 19:02:06.611283871 +0000 UTC m=+1397.101411691" Oct 30 19:02:06.652566 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:06.652529 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 30 19:02:06.652866 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:06.652838 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" podUID="43bf10dc-7107-4803-a386-c98e0a9c3bc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 30 19:02:06.980948 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:06.980857 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:06.982753 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:06.982725 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body= Oct 30 19:02:06.982883 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:06.982775 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" podUID="e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" Oct 30 19:02:07.380183 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:07.380094 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:07.656657 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:07.649993 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 30 19:02:07.656657 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:07.650056 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" podUID="43bf10dc-7107-4803-a386-c98e0a9c3bc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 30 19:02:07.981653 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:07.981170 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body= Oct 30 19:02:07.981653 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:07.981230 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" podUID="e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" Oct 30 19:02:08.642293 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:08.630939 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 30 19:02:08.642293 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:08.631000 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" podUID="43bf10dc-7107-4803-a386-c98e0a9c3bc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 30 19:02:08.982646 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:08.981957 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body= Oct 30 19:02:08.982646 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:08.982023 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" podUID="e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" Oct 30 19:02:09.641598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:09.630954 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 30 19:02:09.641598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:09.631015 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" podUID="43bf10dc-7107-4803-a386-c98e0a9c3bc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 30 19:02:09.981759 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:09.981659 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body= Oct 30 19:02:09.981759 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:09.981738 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" podUID="e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" Oct 30 19:02:10.631611 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:10.630932 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 30 19:02:10.631611 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:10.631025 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" podUID="43bf10dc-7107-4803-a386-c98e0a9c3bc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 30 19:02:10.980998 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:10.980960 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body= Oct 30 19:02:10.981170 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:10.981025 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" podUID="e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" Oct 30 19:02:11.628919 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:11.628880 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" start-of-body= Oct 30 19:02:11.629137 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:11.628955 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" podUID="43bf10dc-7107-4803-a386-c98e0a9c3bc2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.82:15021/healthz/ready\": dial tcp 10.132.0.82:15021: connect: connection refused" Oct 30 19:02:11.981012 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:11.980982 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" start-of-body= Oct 30 19:02:11.981398 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:11.981029 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" podUID="e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.83:15021/healthz/ready\": dial tcp 10.132.0.83:15021: connect: connection refused" Oct 30 19:02:12.419023 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:12.418984 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:12.824468 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:12.824382 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:12.825853 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:12.825828 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:12.827171 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:12.827145 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:02:13.100120 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:13.100029 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:13.100597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:13.100425 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:13.101379 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:13.101351 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:02:17.193594 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:17.193303 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:19.596987 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:19.596922 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:20.118057 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:02:20.118021 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:02:22.199227 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:22.199191 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:22.624802 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:22.624695 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:22.906751 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:22.906699 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:23.032755 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:23.032706 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:23.180527 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:23.180429 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded" Oct 30 19:02:23.296710 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:23.296661 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:23.319118 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:23.319072 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:23.370589 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:23.370532 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded" Oct 30 19:02:23.628762 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:23.628657 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:23.629709 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:23.629687 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 19:02:23.995946 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:23.995897 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded" Oct 30 19:02:23.996919 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:23.996893 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 19:02:24.114991 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:24.113511 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:24.116335 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:24.115426 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 19:02:24.201597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:24.200731 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": context deadline exceeded" Oct 30 19:02:24.201597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:24.201436 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 19:02:24.363654 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:24.359662 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded" Oct 30 19:02:24.383600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:24.379654 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:24.393594 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:24.385786 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:24.393594 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:24.387710 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 19:02:24.393594 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:24.387771 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 19:02:24.393594 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:24.387809 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:02:24.513640 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:24.487941 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:24.748695 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:24.634082 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:24.794320 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:24.792655 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:25.186498 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:25.081820 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/health\": context deadline exceeded" Oct 30 19:02:25.187077 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:25.098345 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded" Oct 30 19:02:25.187602 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:25.120948 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:25.192643 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:25.141530 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:02:25.192843 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:25.192805 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" podUID="0464dd8d-01e7-4ef3-a866-d77fbe31d99a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:25.288036 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:25.224704 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:25.364791 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:25.364746 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded" Oct 30 19:02:25.405709 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:25.405657 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:25.407652 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:25.407617 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:25.470582 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:25.468941 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 19:02:25.493027 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:25.492970 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:25.493791 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:25.493768 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 19:02:25.600633 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:25.572952 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:25.838653 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:25.836130 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/health\": context deadline exceeded" Oct 30 19:02:25.927611 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:25.926001 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/health\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:26.194598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.193729 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:26.197614 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.197042 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:26.228690 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.228647 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:26.278597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.278275 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:26.286267 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.286238 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 19:02:26.413524 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.413468 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:26.415359 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.415336 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9107-fc9c27"] Oct 30 19:02:26.430176 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.428849 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded" Oct 30 19:02:26.510648 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.499175 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:02:26.510648 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.499240 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" podUID="e4b4e21b-a660-4b77-869c-64ec2a9996be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:26.533670 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.533628 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:26.679608 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.664351 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 19:02:26.704691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.704660 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9107-fc9c27"] Oct 30 19:02:26.806866 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.806784 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 19:02:26.853699 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.831719 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 19:02:26.904356 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.665068 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9107-fc9c27" Oct 30 19:02:26.943845 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.943805 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:02:26.945250 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.945217 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" podUID="0133d032-d0fb-4623-8c0e-4ddc8b765c6d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.61:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:26.975009 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.974977 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7v9w4\"" Oct 30 19:02:26.977557 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:26.976663 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vh2v\" (UniqueName: \"kubernetes.io/projected/9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc-kube-api-access-4vh2v\") pod \"test-curl-9107-fc9c27\" (UID: \"9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc\") " pod="opendatahub/test-curl-9107-fc9c27" Oct 30 19:02:27.036922 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:27.036783 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:02:27.107669 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:27.107234 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 19:02:27.110737 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:27.110341 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vh2v\" (UniqueName: \"kubernetes.io/projected/9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc-kube-api-access-4vh2v\") pod \"test-curl-9107-fc9c27\" (UID: \"9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc\") " pod="opendatahub/test-curl-9107-fc9c27" Oct 30 19:02:27.132523 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:27.131727 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 19:02:27.172950 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:27.172915 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vh2v\" (UniqueName: \"kubernetes.io/projected/9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc-kube-api-access-4vh2v\") pod \"test-curl-9107-fc9c27\" (UID: \"9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc\") " pod="opendatahub/test-curl-9107-fc9c27" Oct 30 19:02:27.343004 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:27.342954 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9107-fc9c27" Oct 30 19:02:27.737928 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:27.737893 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9107-fc9c27"] Oct 30 19:02:27.743314 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:02:27.743284 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9d6bbd0a_cb77_4525_b8fc_13749d1a8cdc.slice/crio-c059c1abdee3dffc7c169ef4a311689da3daa69d0196ff19c93e6ffe341d07cb WatchSource:0}: Error finding container c059c1abdee3dffc7c169ef4a311689da3daa69d0196ff19c93e6ffe341d07cb: Status 404 returned error can't find the container with id c059c1abdee3dffc7c169ef4a311689da3daa69d0196ff19c93e6ffe341d07cb Oct 30 19:02:27.745547 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:27.745531 5894 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 19:02:28.299467 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:28.281596 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:28.387598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:28.383536 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9107-fc9c27" event={"ID":"9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc","Type":"ContainerStarted","Data":"c059c1abdee3dffc7c169ef4a311689da3daa69d0196ff19c93e6ffe341d07cb"} Oct 30 19:02:31.404395 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:31.404330 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9107-fc9c27" event={"ID":"9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc","Type":"ContainerStarted","Data":"04dcbf7ae25076682a1770a799323198b786bf3ffcf8841986306db7ad948d60"} Oct 30 19:02:31.519519 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:31.519470 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-9107-fc9c27" podStartSLOduration=2.779110439 podStartE2EDuration="5.51945579s" totalImagesPullingTime="2.740345351s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:31.518220285 +0000 UTC m=+1422.008348109" watchObservedRunningTime="2026-10-30 19:02:31.51945579 +0000 UTC m=+1422.009583609" Oct 30 19:02:32.945335 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:32.945301 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9107-fc9c27"] Oct 30 19:02:32.952128 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:32.952100 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9107-fc9c27"] Oct 30 19:02:33.024271 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:33.013299 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:33.126349 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:33.126316 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9107-c74fcf"] Oct 30 19:02:33.127019 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:33.126998 5894 cpu_manager.go:414] "containerMap: removing container" podUID="9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc" containerName="test-curl-9107-fc9c27" Oct 30 19:02:33.127019 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:33.127017 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc" containerName="test-curl-9107-fc9c27" Oct 30 19:02:33.127318 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:33.127144 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc" containerName="test-curl-9107-fc9c27" Oct 30 19:02:33.130841 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:33.130817 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9107-c74fcf" Oct 30 19:02:33.179176 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:33.179147 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9107-c74fcf"] Oct 30 19:02:33.223988 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:33.223911 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzjbs\" (UniqueName: \"kubernetes.io/projected/97a6aa63-2e34-48c7-b09b-bc63cfd6a891-kube-api-access-hzjbs\") pod \"test-curl-9107-c74fcf\" (UID: \"97a6aa63-2e34-48c7-b09b-bc63cfd6a891\") " pod="opendatahub/test-curl-9107-c74fcf" Oct 30 19:02:33.325466 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:33.325433 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzjbs\" (UniqueName: \"kubernetes.io/projected/97a6aa63-2e34-48c7-b09b-bc63cfd6a891-kube-api-access-hzjbs\") pod \"test-curl-9107-c74fcf\" (UID: \"97a6aa63-2e34-48c7-b09b-bc63cfd6a891\") " pod="opendatahub/test-curl-9107-c74fcf" Oct 30 19:02:33.382826 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:33.382796 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzjbs\" (UniqueName: \"kubernetes.io/projected/97a6aa63-2e34-48c7-b09b-bc63cfd6a891-kube-api-access-hzjbs\") pod \"test-curl-9107-c74fcf\" (UID: \"97a6aa63-2e34-48c7-b09b-bc63cfd6a891\") " pod="opendatahub/test-curl-9107-c74fcf" Oct 30 19:02:33.417270 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:33.417240 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-9107-fc9c27" podUID="9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc" containerName="test-curl-9107-fc9c27" containerID="cri-o://04dcbf7ae25076682a1770a799323198b786bf3ffcf8841986306db7ad948d60" gracePeriod=2 Oct 30 19:02:33.443749 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:33.443723 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9107-c74fcf" Oct 30 19:02:33.468585 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:02:33.468548 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9107-fc9c27\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc" pod="opendatahub/test-curl-9107-fc9c27" Oct 30 19:02:33.824719 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:33.824687 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9107-c74fcf"] Oct 30 19:02:33.825379 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:02:33.825353 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97a6aa63_2e34_48c7_b09b_bc63cfd6a891.slice/crio-864bbb7fc9440b288e62b07c0f720dd29261a718ce3c5c3a5fb917df02c235f1 WatchSource:0}: Error finding container 864bbb7fc9440b288e62b07c0f720dd29261a718ce3c5c3a5fb917df02c235f1: Status 404 returned error can't find the container with id 864bbb7fc9440b288e62b07c0f720dd29261a718ce3c5c3a5fb917df02c235f1 Oct 30 19:02:33.884206 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:33.884183 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:34.069391 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:34.069355 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:34.424788 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:34.424757 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9107-c74fcf" event={"ID":"97a6aa63-2e34-48c7-b09b-bc63cfd6a891","Type":"ContainerStarted","Data":"0a03dd64ca449d45816342683f64f725d0444d8e19f30738e0a465ef05bdee5f"} Oct 30 19:02:34.424955 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:34.424796 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9107-c74fcf" event={"ID":"97a6aa63-2e34-48c7-b09b-bc63cfd6a891","Type":"ContainerStarted","Data":"864bbb7fc9440b288e62b07c0f720dd29261a718ce3c5c3a5fb917df02c235f1"} Oct 30 19:02:34.540039 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:34.539979 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-9107-c74fcf" podStartSLOduration=1.539960952 podStartE2EDuration="1.539960952s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:34.539737092 +0000 UTC m=+1425.029864915" watchObservedRunningTime="2026-10-30 19:02:34.539960952 +0000 UTC m=+1425.030088776" Oct 30 19:02:34.712147 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:34.712065 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:34.904524 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:34.904484 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:35.161392 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:35.161349 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:35.750986 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:35.750803 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9107-fc9c27" Oct 30 19:02:35.919680 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:35.905633 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc-kube-api-access-4vh2v\" (UniqueName: \"kubernetes.io/projected/9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc-kube-api-access-4vh2v\") pod \"9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc\" (UID: \"9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc\") " Oct 30 19:02:35.996651 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:35.993226 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc-kube-api-access-4vh2v" pod "9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc" (UID: "9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc"). InnerVolumeSpecName "kube-api-access-4vh2v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:02:36.022628 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:36.020925 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vh2v\" (UniqueName: \"kubernetes.io/projected/9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc-kube-api-access-4vh2v\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:02:36.269301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:36.269217 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9107-c74fcf"] Oct 30 19:02:36.282298 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:36.282268 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9107-c74fcf"] Oct 30 19:02:36.357612 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:36.345412 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:36.484682 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:36.484503 5894 generic.go:356] "Generic (PLEG): container finished" podID="9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc" containerID="04dcbf7ae25076682a1770a799323198b786bf3ffcf8841986306db7ad948d60" exitCode=137 Oct 30 19:02:36.484881 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:36.484724 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-9107-c74fcf" podUID="97a6aa63-2e34-48c7-b09b-bc63cfd6a891" containerName="test-curl-9107-c74fcf" containerID="cri-o://0a03dd64ca449d45816342683f64f725d0444d8e19f30738e0a465ef05bdee5f" gracePeriod=2 Oct 30 19:02:36.484881 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:36.484861 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9107-fc9c27" Oct 30 19:02:36.485942 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:36.485923 5894 scope.go:122] "RemoveContainer" containerID="04dcbf7ae25076682a1770a799323198b786bf3ffcf8841986306db7ad948d60" Oct 30 19:02:36.519862 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:36.519836 5894 scope.go:122] "RemoveContainer" containerID="04dcbf7ae25076682a1770a799323198b786bf3ffcf8841986306db7ad948d60" Oct 30 19:02:36.520238 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:02:36.520218 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04dcbf7ae25076682a1770a799323198b786bf3ffcf8841986306db7ad948d60\": container with ID starting with 04dcbf7ae25076682a1770a799323198b786bf3ffcf8841986306db7ad948d60 not found: ID does not exist" containerID="04dcbf7ae25076682a1770a799323198b786bf3ffcf8841986306db7ad948d60" Oct 30 19:02:36.520306 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:36.520249 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04dcbf7ae25076682a1770a799323198b786bf3ffcf8841986306db7ad948d60"} err="failed to get container status \"04dcbf7ae25076682a1770a799323198b786bf3ffcf8841986306db7ad948d60\": rpc error: code = NotFound desc = could not find container \"04dcbf7ae25076682a1770a799323198b786bf3ffcf8841986306db7ad948d60\": container with ID starting with 04dcbf7ae25076682a1770a799323198b786bf3ffcf8841986306db7ad948d60 not found: ID does not exist" Oct 30 19:02:36.536681 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:02:36.536507 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9107-c74fcf\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="97a6aa63-2e34-48c7-b09b-bc63cfd6a891" pod="opendatahub/test-curl-9107-c74fcf" Oct 30 19:02:36.598633 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:02:36.591851 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9107-fc9c27\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc" pod="opendatahub/test-curl-9107-fc9c27" Oct 30 19:02:36.642649 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:36.642018 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:36.642840 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:02:36.642707 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9107-c74fcf\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="97a6aa63-2e34-48c7-b09b-bc63cfd6a891" pod="opendatahub/test-curl-9107-c74fcf" Oct 30 19:02:37.296016 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.295987 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g"] Oct 30 19:02:37.296714 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.296694 5894 cpu_manager.go:414] "containerMap: removing container" podUID="97a6aa63-2e34-48c7-b09b-bc63cfd6a891" containerName="test-curl-9107-c74fcf" Oct 30 19:02:37.296823 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.296717 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="97a6aa63-2e34-48c7-b09b-bc63cfd6a891" containerName="test-curl-9107-c74fcf" Oct 30 19:02:37.296885 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.296867 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="97a6aa63-2e34-48c7-b09b-bc63cfd6a891" containerName="test-curl-9107-c74fcf" Oct 30 19:02:37.301368 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.301347 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.352722 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:02:37.352671 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9107-c74fcf\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="97a6aa63-2e34-48c7-b09b-bc63cfd6a891" pod="opendatahub/test-curl-9107-c74fcf" Oct 30 19:02:37.402597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.402549 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-975cf5\"/\"openshift-service-ca.crt\"" Oct 30 19:02:37.402770 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.402615 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-975cf5\"/\"kube-root-ca.crt\"" Oct 30 19:02:37.402770 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.402681 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-975cf5\"/\"distinct-2-w1-975cf5-kserve-self-signed-certs\"" Oct 30 19:02:37.402770 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.402758 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-975cf5\"/\"default-dockercfg-5h568\"" Oct 30 19:02:37.408163 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.408138 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g"] Oct 30 19:02:37.420143 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.420109 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-home\") pod \"distinct-2-w1-975cf5-kserve-b58c88d6b-4429g\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.420259 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.420203 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-model-cache\") pod \"distinct-2-w1-975cf5-kserve-b58c88d6b-4429g\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.420301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.420255 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-tls-certs\") pod \"distinct-2-w1-975cf5-kserve-b58c88d6b-4429g\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.420301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.420287 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-tmp-dir\") pod \"distinct-2-w1-975cf5-kserve-b58c88d6b-4429g\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.420462 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.420443 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwxk6\" (UniqueName: \"kubernetes.io/projected/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-kube-api-access-qwxk6\") pod \"distinct-2-w1-975cf5-kserve-b58c88d6b-4429g\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.420534 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.420492 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-dshm\") pod \"distinct-2-w1-975cf5-kserve-b58c88d6b-4429g\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.438382 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.438349 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:37.524607 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.522491 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-home\") pod \"distinct-2-w1-975cf5-kserve-b58c88d6b-4429g\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.524607 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.522601 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-model-cache\") pod \"distinct-2-w1-975cf5-kserve-b58c88d6b-4429g\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.524607 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.522643 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-tls-certs\") pod \"distinct-2-w1-975cf5-kserve-b58c88d6b-4429g\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.524607 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.522670 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-tmp-dir\") pod \"distinct-2-w1-975cf5-kserve-b58c88d6b-4429g\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.524607 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.522889 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwxk6\" (UniqueName: \"kubernetes.io/projected/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-kube-api-access-qwxk6\") pod \"distinct-2-w1-975cf5-kserve-b58c88d6b-4429g\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.524607 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.522929 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-dshm\") pod \"distinct-2-w1-975cf5-kserve-b58c88d6b-4429g\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.559644 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:02:37.531834 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9107-fc9c27\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc" pod="opendatahub/test-curl-9107-fc9c27" Oct 30 19:02:37.559644 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.532253 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-tmp-dir\") pod \"distinct-2-w1-975cf5-kserve-b58c88d6b-4429g\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.559644 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.532837 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-home\") pod \"distinct-2-w1-975cf5-kserve-b58c88d6b-4429g\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.559644 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.533065 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-model-cache\") pod \"distinct-2-w1-975cf5-kserve-b58c88d6b-4429g\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.562122 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.562091 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-dshm\") pod \"distinct-2-w1-975cf5-kserve-b58c88d6b-4429g\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.586098 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.574783 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-tls-certs\") pod \"distinct-2-w1-975cf5-kserve-b58c88d6b-4429g\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.611138 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.611105 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwxk6\" (UniqueName: \"kubernetes.io/projected/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-kube-api-access-qwxk6\") pod \"distinct-2-w1-975cf5-kserve-b58c88d6b-4429g\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.614150 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.614062 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:37.721592 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:37.720512 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc" path="/var/lib/kubelet/pods/9d6bbd0a-cb77-4525-b8fc-13749d1a8cdc/volumes" Oct 30 19:02:38.122597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:38.118613 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:38.352177 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:38.352083 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g"] Oct 30 19:02:38.358035 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:02:38.358009 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podce4ba09e_cbff_413e_b8cd_17e8de0ce8a2.slice/crio-e96553520cb795f4257f2bfdf951ee53c7a3cc9907532541d158fd59bf63c494 WatchSource:0}: Error finding container e96553520cb795f4257f2bfdf951ee53c7a3cc9907532541d158fd59bf63c494: Status 404 returned error can't find the container with id e96553520cb795f4257f2bfdf951ee53c7a3cc9907532541d158fd59bf63c494 Oct 30 19:02:38.553623 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:38.553562 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" event={"ID":"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2","Type":"ContainerStarted","Data":"a16415535e32f16a18f7423509746f11c704a8a706b04e433e746ca0450af7ca"} Oct 30 19:02:38.553753 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:38.553643 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" event={"ID":"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2","Type":"ContainerStarted","Data":"e96553520cb795f4257f2bfdf951ee53c7a3cc9907532541d158fd59bf63c494"} Oct 30 19:02:38.614364 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:38.614294 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:38.669288 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:38.669242 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" podStartSLOduration=1.669229369 podStartE2EDuration="1.669229369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:38.66602467 +0000 UTC m=+1429.156152493" watchObservedRunningTime="2026-10-30 19:02:38.669229369 +0000 UTC m=+1429.159357189" Oct 30 19:02:38.763141 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:38.763106 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:38.796919 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:38.791902 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9107-c74fcf" Oct 30 19:02:38.894006 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:38.893967 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/97a6aa63-2e34-48c7-b09b-bc63cfd6a891-kube-api-access-hzjbs\" (UniqueName: \"kubernetes.io/projected/97a6aa63-2e34-48c7-b09b-bc63cfd6a891-kube-api-access-hzjbs\") pod \"97a6aa63-2e34-48c7-b09b-bc63cfd6a891\" (UID: \"97a6aa63-2e34-48c7-b09b-bc63cfd6a891\") " Oct 30 19:02:38.896214 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:38.896185 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97a6aa63-2e34-48c7-b09b-bc63cfd6a891-kube-api-access-hzjbs" pod "97a6aa63-2e34-48c7-b09b-bc63cfd6a891" (UID: "97a6aa63-2e34-48c7-b09b-bc63cfd6a891"). InnerVolumeSpecName "kube-api-access-hzjbs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:02:38.996126 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:38.996085 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzjbs\" (UniqueName: \"kubernetes.io/projected/97a6aa63-2e34-48c7-b09b-bc63cfd6a891-kube-api-access-hzjbs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:02:39.402600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:39.397087 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:39.753680 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:39.752089 5894 generic.go:356] "Generic (PLEG): container finished" podID="97a6aa63-2e34-48c7-b09b-bc63cfd6a891" containerID="0a03dd64ca449d45816342683f64f725d0444d8e19f30738e0a465ef05bdee5f" exitCode=137 Oct 30 19:02:39.805749 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:39.805487 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9107-c74fcf" Oct 30 19:02:39.808629 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:39.808588 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="97a6aa63-2e34-48c7-b09b-bc63cfd6a891" path="/var/lib/kubelet/pods/97a6aa63-2e34-48c7-b09b-bc63cfd6a891/volumes" Oct 30 19:02:39.812764 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:39.812133 5894 scope.go:122] "RemoveContainer" containerID="0a03dd64ca449d45816342683f64f725d0444d8e19f30738e0a465ef05bdee5f" Oct 30 19:02:39.829059 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:39.828966 5894 scope.go:122] "RemoveContainer" containerID="0a03dd64ca449d45816342683f64f725d0444d8e19f30738e0a465ef05bdee5f" Oct 30 19:02:39.829406 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:02:39.829377 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a03dd64ca449d45816342683f64f725d0444d8e19f30738e0a465ef05bdee5f\": container with ID starting with 0a03dd64ca449d45816342683f64f725d0444d8e19f30738e0a465ef05bdee5f not found: ID does not exist" containerID="0a03dd64ca449d45816342683f64f725d0444d8e19f30738e0a465ef05bdee5f" Oct 30 19:02:39.829538 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:39.829416 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a03dd64ca449d45816342683f64f725d0444d8e19f30738e0a465ef05bdee5f"} err="failed to get container status \"0a03dd64ca449d45816342683f64f725d0444d8e19f30738e0a465ef05bdee5f\": rpc error: code = NotFound desc = could not find container \"0a03dd64ca449d45816342683f64f725d0444d8e19f30738e0a465ef05bdee5f\": container with ID starting with 0a03dd64ca449d45816342683f64f725d0444d8e19f30738e0a465ef05bdee5f not found: ID does not exist" Oct 30 19:02:40.066924 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:40.066820 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:40.249750 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:40.249715 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:41.143067 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:41.143036 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:41.451841 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:41.451759 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:41.640165 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:41.632344 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:42.050609 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.048473 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz"] Oct 30 19:02:42.067897 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.067864 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.122250 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.122215 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz"] Oct 30 19:02:42.155556 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.154858 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-4b66bd06-openshift-default-dockercfg-bbfcs\"" Oct 30 19:02:42.239336 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.229715 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-credential-socket\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.240972 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.240940 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-data\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.241768 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.241742 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istiod-ca-cert\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.241986 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.241970 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-workload-socket\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.242125 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.242106 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-token\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.242273 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.242257 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cn9lj\" (UniqueName: \"kubernetes.io/projected/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-kube-api-access-cn9lj\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.242422 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.242407 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-workload-certs\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.242724 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.242704 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-envoy\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.243675 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.243652 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-podinfo\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.243940 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.241611 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:42.345100 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.345003 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-envoy\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.345100 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.345068 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-podinfo\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.345325 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.345135 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-credential-socket\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.345325 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.345211 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-data\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.345325 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.345247 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istiod-ca-cert\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.345477 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.345332 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-workload-socket\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.345477 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.345368 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-token\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.345477 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.345411 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cn9lj\" (UniqueName: \"kubernetes.io/projected/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-kube-api-access-cn9lj\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.345638 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.345477 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-workload-certs\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.345770 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.345732 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-credential-socket\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.345952 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.345927 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-workload-certs\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.346097 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.346074 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istiod-ca-cert\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.346550 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.346524 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-workload-socket\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.346761 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.346734 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-data\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.348044 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.348021 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-envoy\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.348719 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.348697 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-podinfo\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.404873 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.404836 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-token\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.405069 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.405045 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cn9lj\" (UniqueName: \"kubernetes.io/projected/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-kube-api-access-cn9lj\") pod \"e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.527828 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.523500 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:42.721391 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.721062 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:42.753426 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.750504 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:42.949619 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:42.949561 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:43.736786 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:43.736758 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz"] Oct 30 19:02:43.737852 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:02:43.737813 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45d20e5e_bffc_4bf1_a51a_592f6c395eb4.slice/crio-27e6b4618115ada0189901f481e8e9308ce94cf27f64008fe1d04e200120d387 WatchSource:0}: Error finding container 27e6b4618115ada0189901f481e8e9308ce94cf27f64008fe1d04e200120d387: Status 404 returned error can't find the container with id 27e6b4618115ada0189901f481e8e9308ce94cf27f64008fe1d04e200120d387 Oct 30 19:02:43.740757 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:43.740714 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:02:43.740858 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:43.740794 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:02:43.740858 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:43.740836 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:02:43.852255 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:43.850912 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" event={"ID":"45d20e5e-bffc-4bf1-a51a-592f6c395eb4","Type":"ContainerStarted","Data":"d48c568c10c6ba1eb995553d22df2bd8e319c6888d2dd1ef6cb2face1d9c0c26"} Oct 30 19:02:43.852255 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:43.850978 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" event={"ID":"45d20e5e-bffc-4bf1-a51a-592f6c395eb4","Type":"ContainerStarted","Data":"27e6b4618115ada0189901f481e8e9308ce94cf27f64008fe1d04e200120d387"} Oct 30 19:02:44.021306 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:44.021194 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" podStartSLOduration=3.021176089 podStartE2EDuration="3.021176089s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:02:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:02:44.018998077 +0000 UTC m=+1434.509125900" watchObservedRunningTime="2026-10-30 19:02:44.021176089 +0000 UTC m=+1434.511303911" Oct 30 19:02:44.130099 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:44.130068 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:44.319841 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:44.319755 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:44.501413 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:44.501383 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:44.726642 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:44.726600 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:44.731802 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:44.731777 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:44.961684 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:44.961653 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:44.963613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:44.963592 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:02:46.719299 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:46.719264 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:46.903244 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:46.903214 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:47.087631 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:47.087547 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:47.467054 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:47.467014 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:47.655898 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:47.655863 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:47.669188 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:47.669158 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:47.684855 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:47.684828 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:48.221888 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:48.179935 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:49.187632 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:49.187596 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:50.118186 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:02:50.118150 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:02:52.356742 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:52.356663 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:52.624697 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:52.621900 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:54.071343 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:54.071265 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:54.278498 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:54.278459 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:54.457269 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:54.457231 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:55.878188 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:55.878153 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:56.127365 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:56.127189 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:57.199780 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:57.199743 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:57.384027 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:57.383995 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:02:58.071600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:58.068760 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" podUID="ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded" Oct 30 19:02:58.191739 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:58.191688 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded" Oct 30 19:02:58.192602 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:58.192319 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded" Oct 30 19:02:58.253239 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:58.253197 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:58.318530 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:58.318486 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:58.429431 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:58.429095 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:58.429662 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:58.429455 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded" Oct 30 19:02:58.560598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:58.559665 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded" Oct 30 19:02:58.620600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:58.617263 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" podUID="ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/health\": context deadline exceeded" Oct 30 19:02:58.645601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:58.635085 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:58.827804 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:58.827707 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:02:58.885598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:58.881672 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" podUID="43b1379e-42d3-4913-ab29-d766689a4a90" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:59.295599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:59.176768 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" podUID="ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:59.295599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:59.239309 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:02:59.295599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:59.251544 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded" Oct 30 19:02:59.496604 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:59.320746 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:59.496604 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:59.495670 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 19:02:59.496604 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:59.495766 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 19:02:59.561032 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:59.554034 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:02:59.561032 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:59.554108 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" podUID="f247ccba-3633-4301-b8f2-41a0a8eab7ea" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:59.561032 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:59.320833 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded" Oct 30 19:02:59.561032 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:59.464743 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded" Oct 30 19:02:59.597261 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:59.593771 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 19:02:59.597261 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:59.593879 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 19:02:59.597261 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:59.323869 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:59.620626 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:59.605208 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 19:02:59.620626 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:59.464700 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:02:59.620626 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:59.606713 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:02:59.674484 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:59.631206 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": context deadline exceeded" Oct 30 19:02:59.674484 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:59.632499 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 19:02:59.674484 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:02:59.672649 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded" Oct 30 19:03:00.031736 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:00.023850 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:03:00.044249 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:00.044205 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" podUID="5a3f5720-0f19-4f6d-9609-1924e6d45b3e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:03:00.063597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:00.062074 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 19:03:00.201596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:00.178253 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" podUID="ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded" Oct 30 19:03:00.595599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:00.474642 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": context deadline exceeded" Oct 30 19:03:00.668598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:00.624696 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:03:00.668598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:00.624733 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:03:00.668849 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:00.624762 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded" Oct 30 19:03:00.692616 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:00.624781 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded" Oct 30 19:03:00.692616 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:00.624800 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": context deadline exceeded" Oct 30 19:03:00.692616 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:00.689529 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded" Oct 30 19:03:00.692616 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:00.663729 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:03:01.292615 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:01.280991 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 19:03:01.292615 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:01.281307 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" podUID="ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded" Oct 30 19:03:01.292615 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:01.286276 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 19:03:01.324032 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:01.324005 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 19:03:01.386655 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:01.372214 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:03:01.386655 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:01.374040 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 19:03:01.440606 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:01.425116 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 19:03:01.470426 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:01.470392 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 19:03:01.515964 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:01.515932 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:03:01.524937 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:01.524913 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 19:03:01.802362 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:01.802322 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:05.283516 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:05.283474 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:05.467479 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:05.467450 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:06.578816 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:06.578781 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:10.783395 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:10.783363 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:11.194462 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:11.194426 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:12.929531 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:12.929495 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:15.867776 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:15.867745 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:16.427862 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:16.427824 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:18.752377 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:18.752342 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:19.932431 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:19.932401 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:20.118685 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:03:20.118652 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:03:20.274797 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:20.274727 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk"] Oct 30 19:03:20.277653 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:20.277636 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" Oct 30 19:03:20.360751 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:20.360719 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf26fd33-d037-4630-a87d-ee7487a05eb0-openshift-service-ca\") pod \"maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk\" (UID: \"cf26fd33-d037-4630-a87d-ee7487a05eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" Oct 30 19:03:20.360910 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:20.360874 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf26fd33-d037-4630-a87d-ee7487a05eb0-metrics-certs\") pod \"maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk\" (UID: \"cf26fd33-d037-4630-a87d-ee7487a05eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" Oct 30 19:03:20.360910 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:20.360900 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl657\" (UniqueName: \"kubernetes.io/projected/cf26fd33-d037-4630-a87d-ee7487a05eb0-kube-api-access-hl657\") pod \"maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk\" (UID: \"cf26fd33-d037-4630-a87d-ee7487a05eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" Oct 30 19:03:20.361001 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:20.360973 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf26fd33-d037-4630-a87d-ee7487a05eb0-maas-api-tls\") pod \"maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk\" (UID: \"cf26fd33-d037-4630-a87d-ee7487a05eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" Oct 30 19:03:20.381800 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:20.381772 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-cfaf8925-serving-cert\"" Oct 30 19:03:20.385180 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:20.385159 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk"] Oct 30 19:03:20.462361 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:20.462332 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf26fd33-d037-4630-a87d-ee7487a05eb0-openshift-service-ca\") pod \"maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk\" (UID: \"cf26fd33-d037-4630-a87d-ee7487a05eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" Oct 30 19:03:20.462535 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:20.462482 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf26fd33-d037-4630-a87d-ee7487a05eb0-metrics-certs\") pod \"maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk\" (UID: \"cf26fd33-d037-4630-a87d-ee7487a05eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" Oct 30 19:03:20.462535 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:20.462523 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl657\" (UniqueName: \"kubernetes.io/projected/cf26fd33-d037-4630-a87d-ee7487a05eb0-kube-api-access-hl657\") pod \"maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk\" (UID: \"cf26fd33-d037-4630-a87d-ee7487a05eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" Oct 30 19:03:20.462676 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:20.462603 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf26fd33-d037-4630-a87d-ee7487a05eb0-maas-api-tls\") pod \"maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk\" (UID: \"cf26fd33-d037-4630-a87d-ee7487a05eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" Oct 30 19:03:20.462733 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:03:20.462704 5894 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-serving-cert: secret "maas-api-e2e-migrate-cfaf8925-serving-cert" not found Oct 30 19:03:20.462786 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:03:20.462775 5894 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cf26fd33-d037-4630-a87d-ee7487a05eb0-maas-api-tls podName:cf26fd33-d037-4630-a87d-ee7487a05eb0 nodeName:}" failed. No retries permitted until 2026-10-30 19:03:20.962753446 +0000 UTC m=+1471.452881259 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/cf26fd33-d037-4630-a87d-ee7487a05eb0-maas-api-tls") pod "maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" (UID: "cf26fd33-d037-4630-a87d-ee7487a05eb0") : secret "maas-api-e2e-migrate-cfaf8925-serving-cert" not found Oct 30 19:03:20.463078 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:20.463059 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf26fd33-d037-4630-a87d-ee7487a05eb0-openshift-service-ca\") pod \"maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk\" (UID: \"cf26fd33-d037-4630-a87d-ee7487a05eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" Oct 30 19:03:20.464920 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:20.464903 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf26fd33-d037-4630-a87d-ee7487a05eb0-metrics-certs\") pod \"maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk\" (UID: \"cf26fd33-d037-4630-a87d-ee7487a05eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" Oct 30 19:03:20.511674 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:20.511647 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:20.562651 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:20.562512 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl657\" (UniqueName: \"kubernetes.io/projected/cf26fd33-d037-4630-a87d-ee7487a05eb0-kube-api-access-hl657\") pod \"maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk\" (UID: \"cf26fd33-d037-4630-a87d-ee7487a05eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" Oct 30 19:03:20.703784 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:20.703735 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:20.975607 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:20.975553 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf26fd33-d037-4630-a87d-ee7487a05eb0-maas-api-tls\") pod \"maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk\" (UID: \"cf26fd33-d037-4630-a87d-ee7487a05eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" Oct 30 19:03:20.978195 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:20.978166 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf26fd33-d037-4630-a87d-ee7487a05eb0-maas-api-tls\") pod \"maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk\" (UID: \"cf26fd33-d037-4630-a87d-ee7487a05eb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" Oct 30 19:03:21.115280 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.115247 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:21.153973 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.153943 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm"] Oct 30 19:03:21.157951 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.157925 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.177946 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.177918 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-istio-envoy\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.178064 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.178034 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-istio-data\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.178118 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.178082 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hm2bg\" (UniqueName: \"kubernetes.io/projected/6364da0d-986e-4551-ae12-b4f3f6963799-kube-api-access-hm2bg\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.178118 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.178109 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6364da0d-986e-4551-ae12-b4f3f6963799-istio-podinfo\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.178193 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.178164 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-workload-certs\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.178275 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.178253 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-credential-socket\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.178322 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.178311 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-workload-socket\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.178359 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.178343 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6364da0d-986e-4551-ae12-b4f3f6963799-istio-token\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.178392 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.178370 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6364da0d-986e-4551-ae12-b4f3f6963799-istiod-ca-cert\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.190147 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.190124 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" Oct 30 19:03:21.231845 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.231679 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm"] Oct 30 19:03:21.280538 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.280504 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-workload-socket\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.280758 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.280590 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6364da0d-986e-4551-ae12-b4f3f6963799-istio-token\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.280758 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.280637 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6364da0d-986e-4551-ae12-b4f3f6963799-istiod-ca-cert\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.280758 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.280711 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-istio-envoy\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.280932 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.280821 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-istio-data\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.280932 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.280885 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hm2bg\" (UniqueName: \"kubernetes.io/projected/6364da0d-986e-4551-ae12-b4f3f6963799-kube-api-access-hm2bg\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.281038 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.280931 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6364da0d-986e-4551-ae12-b4f3f6963799-istio-podinfo\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.281038 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.280988 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-workload-certs\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.281142 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.281036 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-workload-socket\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.281142 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.281049 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-credential-socket\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.281367 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.281342 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-istio-data\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.282844 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.282245 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6364da0d-986e-4551-ae12-b4f3f6963799-istiod-ca-cert\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.282844 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.282523 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-workload-certs\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.282844 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.282804 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-credential-socket\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.286790 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.286768 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-istio-envoy\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.287640 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.287620 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6364da0d-986e-4551-ae12-b4f3f6963799-istio-podinfo\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.338827 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.338791 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hm2bg\" (UniqueName: \"kubernetes.io/projected/6364da0d-986e-4551-ae12-b4f3f6963799-kube-api-access-hm2bg\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.339016 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.338942 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6364da0d-986e-4551-ae12-b4f3f6963799-istio-token\") pod \"e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.472077 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.472042 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:21.577785 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.576541 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk"] Oct 30 19:03:21.579567 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:03:21.579532 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf26fd33_d037_4630_a87d_ee7487a05eb0.slice/crio-afe185dd90493475f99035ead83af9358f8467b9b7f2f717a8f372c3bcac2d57 WatchSource:0}: Error finding container afe185dd90493475f99035ead83af9358f8467b9b7f2f717a8f372c3bcac2d57: Status 404 returned error can't find the container with id afe185dd90493475f99035ead83af9358f8467b9b7f2f717a8f372c3bcac2d57 Oct 30 19:03:21.862244 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.862204 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm"] Oct 30 19:03:21.862555 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:03:21.862530 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6364da0d_986e_4551_ae12_b4f3f6963799.slice/crio-24f401004935d5a65c764b69bfbd3ef5b16df7af83bb460122e3b76846af5d3c WatchSource:0}: Error finding container 24f401004935d5a65c764b69bfbd3ef5b16df7af83bb460122e3b76846af5d3c: Status 404 returned error can't find the container with id 24f401004935d5a65c764b69bfbd3ef5b16df7af83bb460122e3b76846af5d3c Oct 30 19:03:21.865384 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.865351 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:03:21.865543 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.865417 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:03:21.865543 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:21.865444 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:03:22.490648 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:22.488283 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" event={"ID":"6364da0d-986e-4551-ae12-b4f3f6963799","Type":"ContainerStarted","Data":"3694dbd41ad629bf0d2638e0870bd49a522215392fd4a54a9e1aee6257ed781d"} Oct 30 19:03:22.490648 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:22.488351 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" event={"ID":"6364da0d-986e-4551-ae12-b4f3f6963799","Type":"ContainerStarted","Data":"24f401004935d5a65c764b69bfbd3ef5b16df7af83bb460122e3b76846af5d3c"} Oct 30 19:03:22.494122 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:22.493205 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" event={"ID":"cf26fd33-d037-4630-a87d-ee7487a05eb0","Type":"ContainerStarted","Data":"afe185dd90493475f99035ead83af9358f8467b9b7f2f717a8f372c3bcac2d57"} Oct 30 19:03:22.615261 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:22.615185 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" podStartSLOduration=1.61516387 podStartE2EDuration="1.61516387s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:22.613529574 +0000 UTC m=+1473.103657396" watchObservedRunningTime="2026-10-30 19:03:22.61516387 +0000 UTC m=+1473.105291690" Oct 30 19:03:23.472553 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:23.472519 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:23.474317 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:23.474292 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body= Oct 30 19:03:23.474426 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:23.474349 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" podUID="6364da0d-986e-4551-ae12-b4f3f6963799" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" Oct 30 19:03:23.501243 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:23.501210 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" event={"ID":"cf26fd33-d037-4630-a87d-ee7487a05eb0","Type":"ContainerStarted","Data":"089c359b45ba670f9b1f2c7eec1612b5cf89be5a97c7bd71b73375d5255c9d98"} Oct 30 19:03:23.501606 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:23.501398 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" Oct 30 19:03:23.658989 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:23.658924 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" podStartSLOduration=2.7562809550000003 podStartE2EDuration="3.658908757s" totalImagesPullingTime="902.627802ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:23.656875606 +0000 UTC m=+1474.147003436" watchObservedRunningTime="2026-10-30 19:03:23.658908757 +0000 UTC m=+1474.149036641" Oct 30 19:03:24.472903 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:24.472868 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body= Oct 30 19:03:24.473091 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:24.472917 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" podUID="6364da0d-986e-4551-ae12-b4f3f6963799" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" Oct 30 19:03:25.319523 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:25.319435 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:25.492069 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:25.491978 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body= Oct 30 19:03:25.492069 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:25.492041 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" podUID="6364da0d-986e-4551-ae12-b4f3f6963799" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" Oct 30 19:03:26.472529 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:26.472491 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body= Oct 30 19:03:26.472987 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:26.472563 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" podUID="6364da0d-986e-4551-ae12-b4f3f6963799" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" Oct 30 19:03:27.472728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:27.472694 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body= Oct 30 19:03:27.473217 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:27.472745 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" podUID="6364da0d-986e-4551-ae12-b4f3f6963799" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" Oct 30 19:03:28.472376 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:28.472344 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" start-of-body= Oct 30 19:03:28.472615 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:28.472395 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" podUID="6364da0d-986e-4551-ae12-b4f3f6963799" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": dial tcp 10.132.0.89:15021: connect: connection refused" Oct 30 19:03:29.544597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:29.538809 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" Oct 30 19:03:30.310943 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:30.310912 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:30.312836 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:30.312760 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:30.318844 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:30.318822 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:03:30.470665 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:30.467976 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:30.739597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:30.738800 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:31.157124 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:31.157086 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:35.862276 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:35.859098 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:40.227055 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.227021 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m"] Oct 30 19:03:40.231653 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.231630 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" Oct 30 19:03:40.333985 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.333956 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-4b66bd06-serving-cert\"" Oct 30 19:03:40.337413 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.337385 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m"] Oct 30 19:03:40.356652 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.356619 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rqv68\" (UniqueName: \"kubernetes.io/projected/450f937c-7398-4133-b141-18a3ae03f31c-kube-api-access-rqv68\") pod \"maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m\" (UID: \"450f937c-7398-4133-b141-18a3ae03f31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" Oct 30 19:03:40.356797 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.356713 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/450f937c-7398-4133-b141-18a3ae03f31c-openshift-service-ca\") pod \"maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m\" (UID: \"450f937c-7398-4133-b141-18a3ae03f31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" Oct 30 19:03:40.356903 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.356879 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/450f937c-7398-4133-b141-18a3ae03f31c-maas-api-tls\") pod \"maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m\" (UID: \"450f937c-7398-4133-b141-18a3ae03f31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" Oct 30 19:03:40.356966 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.356952 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/450f937c-7398-4133-b141-18a3ae03f31c-metrics-certs\") pod \"maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m\" (UID: \"450f937c-7398-4133-b141-18a3ae03f31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" Oct 30 19:03:40.458359 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.458321 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rqv68\" (UniqueName: \"kubernetes.io/projected/450f937c-7398-4133-b141-18a3ae03f31c-kube-api-access-rqv68\") pod \"maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m\" (UID: \"450f937c-7398-4133-b141-18a3ae03f31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" Oct 30 19:03:40.458359 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.458363 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/450f937c-7398-4133-b141-18a3ae03f31c-openshift-service-ca\") pod \"maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m\" (UID: \"450f937c-7398-4133-b141-18a3ae03f31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" Oct 30 19:03:40.458682 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.458507 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/450f937c-7398-4133-b141-18a3ae03f31c-maas-api-tls\") pod \"maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m\" (UID: \"450f937c-7398-4133-b141-18a3ae03f31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" Oct 30 19:03:40.458682 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.458588 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/450f937c-7398-4133-b141-18a3ae03f31c-metrics-certs\") pod \"maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m\" (UID: \"450f937c-7398-4133-b141-18a3ae03f31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" Oct 30 19:03:40.458819 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:03:40.458782 5894 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-serving-cert: secret "maas-api-e2e-ait-4b66bd06-serving-cert" not found Oct 30 19:03:40.458882 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:03:40.458849 5894 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/450f937c-7398-4133-b141-18a3ae03f31c-maas-api-tls podName:450f937c-7398-4133-b141-18a3ae03f31c nodeName:}" failed. No retries permitted until 2026-10-30 19:03:40.958832314 +0000 UTC m=+1491.448960113 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/450f937c-7398-4133-b141-18a3ae03f31c-maas-api-tls") pod "maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" (UID: "450f937c-7398-4133-b141-18a3ae03f31c") : secret "maas-api-e2e-ait-4b66bd06-serving-cert" not found Oct 30 19:03:40.459325 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.459301 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/450f937c-7398-4133-b141-18a3ae03f31c-openshift-service-ca\") pod \"maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m\" (UID: \"450f937c-7398-4133-b141-18a3ae03f31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" Oct 30 19:03:40.496643 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.492934 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/450f937c-7398-4133-b141-18a3ae03f31c-metrics-certs\") pod \"maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m\" (UID: \"450f937c-7398-4133-b141-18a3ae03f31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" Oct 30 19:03:40.501184 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.501069 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:40.573947 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.573906 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rqv68\" (UniqueName: \"kubernetes.io/projected/450f937c-7398-4133-b141-18a3ae03f31c-kube-api-access-rqv68\") pod \"maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m\" (UID: \"450f937c-7398-4133-b141-18a3ae03f31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" Oct 30 19:03:40.742557 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.742509 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:40.914404 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.914370 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:40.990021 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.989988 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/450f937c-7398-4133-b141-18a3ae03f31c-maas-api-tls\") pod \"maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m\" (UID: \"450f937c-7398-4133-b141-18a3ae03f31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" Oct 30 19:03:40.992467 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:40.992446 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/450f937c-7398-4133-b141-18a3ae03f31c-maas-api-tls\") pod \"maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m\" (UID: \"450f937c-7398-4133-b141-18a3ae03f31c\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" Oct 30 19:03:41.147621 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:41.147557 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" Oct 30 19:03:41.524156 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:41.524124 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m"] Oct 30 19:03:41.525219 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:03:41.525194 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod450f937c_7398_4133_b141_18a3ae03f31c.slice/crio-8bd64f7cb2bd971fe9f6cc8eebc32989e45ab9a029519e4b5e0d84b78a17bbc7 WatchSource:0}: Error finding container 8bd64f7cb2bd971fe9f6cc8eebc32989e45ab9a029519e4b5e0d84b78a17bbc7: Status 404 returned error can't find the container with id 8bd64f7cb2bd971fe9f6cc8eebc32989e45ab9a029519e4b5e0d84b78a17bbc7 Oct 30 19:03:41.862241 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:41.862160 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" event={"ID":"450f937c-7398-4133-b141-18a3ae03f31c","Type":"ContainerStarted","Data":"8bd64f7cb2bd971fe9f6cc8eebc32989e45ab9a029519e4b5e0d84b78a17bbc7"} Oct 30 19:03:42.869633 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:42.869549 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" event={"ID":"450f937c-7398-4133-b141-18a3ae03f31c","Type":"ContainerStarted","Data":"9224a733595d4995141ce1babede88af6686d63ab8f0f44683dd270964281945"} Oct 30 19:03:42.870014 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:42.869639 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" Oct 30 19:03:42.984297 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:42.984244 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" podStartSLOduration=2.093944035 podStartE2EDuration="2.98422802s" totalImagesPullingTime="890.283985ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:03:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:03:42.983198001 +0000 UTC m=+1493.473325822" watchObservedRunningTime="2026-10-30 19:03:42.98422802 +0000 UTC m=+1493.474355841" Oct 30 19:03:43.330963 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:43.330921 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:43.518631 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:43.518595 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:43.799847 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:43.774701 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:45.477680 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:45.477645 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:48.881507 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:48.881478 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" Oct 30 19:03:49.710855 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:49.710816 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:49.832835 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:49.832804 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 19:03:49.835688 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:49.835663 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 19:03:49.917754 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:49.917719 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:50.120270 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:03:50.120186 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:03:50.211595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:50.207089 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:50.383060 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:50.382980 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:55.058814 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:55.058687 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:59.121230 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:59.121199 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:59.297471 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:59.297439 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:03:59.709922 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:03:59.709883 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:00.177343 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.177312 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q"] Oct 30 19:04:00.183061 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.183037 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.235109 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.235082 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-qwc8z\"" Oct 30 19:04:00.288891 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.288858 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b631fda-764d-4310-9ab8-352947f10ee4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.289070 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.288906 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.289070 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.289040 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.289151 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.289082 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b631fda-764d-4310-9ab8-352947f10ee4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.289151 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.289117 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.289223 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.289156 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lwvxw\" (UniqueName: \"kubernetes.io/projected/7b631fda-764d-4310-9ab8-352947f10ee4-kube-api-access-lwvxw\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.289267 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.289258 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.289325 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.289311 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b631fda-764d-4310-9ab8-352947f10ee4-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.289466 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.289452 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.290396 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.290373 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q"] Oct 30 19:04:00.390988 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.390956 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.391149 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.391062 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b631fda-764d-4310-9ab8-352947f10ee4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.391149 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.391085 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.391149 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.391118 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.391306 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.391148 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b631fda-764d-4310-9ab8-352947f10ee4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.391306 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.391185 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.391306 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.391228 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lwvxw\" (UniqueName: \"kubernetes.io/projected/7b631fda-764d-4310-9ab8-352947f10ee4-kube-api-access-lwvxw\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.391467 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.391343 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.391467 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.391398 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b631fda-764d-4310-9ab8-352947f10ee4-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.391618 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.391507 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.391618 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.391541 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.391754 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.391625 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.391842 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.391821 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.392098 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.392076 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b631fda-764d-4310-9ab8-352947f10ee4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.393306 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.393279 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.393716 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.393696 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b631fda-764d-4310-9ab8-352947f10ee4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.448700 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.448620 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b631fda-764d-4310-9ab8-352947f10ee4-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.448857 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.448624 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lwvxw\" (UniqueName: \"kubernetes.io/projected/7b631fda-764d-4310-9ab8-352947f10ee4-kube-api-access-lwvxw\") pod \"maas-default-gateway-openshift-default-6db586dd6c-4vm2q\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.496211 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.496179 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:00.930003 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:04:00.929974 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b631fda_764d_4310_9ab8_352947f10ee4.slice/crio-4e17b02e70e5eab54d14523fa06a9b8fdc52e71818372415f566a600c1fa136c WatchSource:0}: Error finding container 4e17b02e70e5eab54d14523fa06a9b8fdc52e71818372415f566a600c1fa136c: Status 404 returned error can't find the container with id 4e17b02e70e5eab54d14523fa06a9b8fdc52e71818372415f566a600c1fa136c Oct 30 19:04:00.930150 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.930122 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q"] Oct 30 19:04:00.932505 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.932478 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:04:00.932594 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.932535 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:04:00.932594 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:00.932565 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:04:01.043810 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:01.043770 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" event={"ID":"7b631fda-764d-4310-9ab8-352947f10ee4","Type":"ContainerStarted","Data":"e463faf08a0b9e0144d9aa9c76416529a1e4c2d7ab0b189421b98a5191b86d07"} Oct 30 19:04:01.043975 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:01.043842 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" event={"ID":"7b631fda-764d-4310-9ab8-352947f10ee4","Type":"ContainerStarted","Data":"4e17b02e70e5eab54d14523fa06a9b8fdc52e71818372415f566a600c1fa136c"} Oct 30 19:04:01.160349 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:01.160288 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" podStartSLOduration=1.160274614 podStartE2EDuration="1.160274614s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:04:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:04:01.159746466 +0000 UTC m=+1511.649874287" watchObservedRunningTime="2026-10-30 19:04:01.160274614 +0000 UTC m=+1511.650402435" Oct 30 19:04:01.496906 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:01.496876 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:01.498284 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:01.498260 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-4vm2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 30 19:04:01.498421 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:01.498311 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" podUID="7b631fda-764d-4310-9ab8-352947f10ee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 30 19:04:02.496937 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:02.496897 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-4vm2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 30 19:04:02.497327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:02.496949 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" podUID="7b631fda-764d-4310-9ab8-352947f10ee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 30 19:04:02.614230 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:02.614195 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:03.528455 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:03.528422 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-4vm2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 30 19:04:03.529240 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:03.529203 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" podUID="7b631fda-764d-4310-9ab8-352947f10ee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 30 19:04:03.544344 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:03.544306 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:04.100988 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:04.100951 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:04.497663 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:04.497627 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-4vm2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 30 19:04:04.497830 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:04.497685 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" podUID="7b631fda-764d-4310-9ab8-352947f10ee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 30 19:04:04.977599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:04.974295 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:05.496791 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:05.496754 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-4vm2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 30 19:04:05.496970 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:05.496820 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" podUID="7b631fda-764d-4310-9ab8-352947f10ee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 30 19:04:06.496884 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:06.496851 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-4vm2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 30 19:04:06.497284 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:06.496901 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" podUID="7b631fda-764d-4310-9ab8-352947f10ee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 30 19:04:07.497438 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:07.497399 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-4vm2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 30 19:04:07.497868 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:07.497466 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" podUID="7b631fda-764d-4310-9ab8-352947f10ee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 30 19:04:08.496479 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:08.496447 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-4vm2q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" start-of-body= Oct 30 19:04:08.496686 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:08.496506 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" podUID="7b631fda-764d-4310-9ab8-352947f10ee4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.91:15021/healthz/ready\": dial tcp 10.132.0.91:15021: connect: connection refused" Oct 30 19:04:08.675645 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:08.675610 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:08.873513 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:08.873428 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:09.062826 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:09.062784 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:10.051535 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:10.051503 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:10.052111 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:10.052084 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:10.052991 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:10.052971 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:04:10.173552 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:10.173520 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:13.011205 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:13.011170 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:13.309335 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:13.309209 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:13.339430 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:13.339364 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj"] Oct 30 19:04:13.345297 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:13.345271 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj" Oct 30 19:04:13.419086 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:13.419052 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj"] Oct 30 19:04:13.443622 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:13.443587 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w3-36d676-dockercfg-mgbt4\"" Oct 30 19:04:13.467647 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:13.443892 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w3-36d676\"" Oct 30 19:04:13.676827 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:13.642285 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kg2xw\" (UniqueName: \"kubernetes.io/projected/3df940b4-b684-4a85-ae0c-2fe4ffe51849-kube-api-access-kg2xw\") pod \"payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj\" (UID: \"3df940b4-b684-4a85-ae0c-2fe4ffe51849\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj" Oct 30 19:04:13.779791 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:13.777877 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3df940b4-b684-4a85-ae0c-2fe4ffe51849-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj\" (UID: \"3df940b4-b684-4a85-ae0c-2fe4ffe51849\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj" Oct 30 19:04:13.879929 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:13.879895 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kg2xw\" (UniqueName: \"kubernetes.io/projected/3df940b4-b684-4a85-ae0c-2fe4ffe51849-kube-api-access-kg2xw\") pod \"payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj\" (UID: \"3df940b4-b684-4a85-ae0c-2fe4ffe51849\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj" Oct 30 19:04:13.880115 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:13.880037 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3df940b4-b684-4a85-ae0c-2fe4ffe51849-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj\" (UID: \"3df940b4-b684-4a85-ae0c-2fe4ffe51849\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj" Oct 30 19:04:13.880706 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:13.880685 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3df940b4-b684-4a85-ae0c-2fe4ffe51849-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj\" (UID: \"3df940b4-b684-4a85-ae0c-2fe4ffe51849\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj" Oct 30 19:04:13.937664 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:13.937568 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kg2xw\" (UniqueName: \"kubernetes.io/projected/3df940b4-b684-4a85-ae0c-2fe4ffe51849-kube-api-access-kg2xw\") pod \"payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj\" (UID: \"3df940b4-b684-4a85-ae0c-2fe4ffe51849\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj" Oct 30 19:04:14.029158 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:14.029124 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj" Oct 30 19:04:14.409225 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:14.409203 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj"] Oct 30 19:04:14.410374 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:04:14.410347 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3df940b4_b684_4a85_ae0c_2fe4ffe51849.slice/crio-d666849e4c7313ff207bf3ff21e39921a5d7ff3620d4675fa0bbc150b104f6b1 WatchSource:0}: Error finding container d666849e4c7313ff207bf3ff21e39921a5d7ff3620d4675fa0bbc150b104f6b1: Status 404 returned error can't find the container with id d666849e4c7313ff207bf3ff21e39921a5d7ff3620d4675fa0bbc150b104f6b1 Oct 30 19:04:15.290158 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:15.290122 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:15.398652 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:15.397469 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj" event={"ID":"3df940b4-b684-4a85-ae0c-2fe4ffe51849","Type":"ContainerStarted","Data":"d666849e4c7313ff207bf3ff21e39921a5d7ff3620d4675fa0bbc150b104f6b1"} Oct 30 19:04:17.414112 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:17.414037 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj" event={"ID":"3df940b4-b684-4a85-ae0c-2fe4ffe51849","Type":"ContainerStarted","Data":"59402d10975d07677e34bf3f5b0248eea4df1908f48f0b1ae3f3b76b0a4f7a70"} Oct 30 19:04:17.414470 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:17.414129 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj" Oct 30 19:04:17.530796 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:17.530736 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj" podStartSLOduration=1.7928483050000001 podStartE2EDuration="4.530719197s" totalImagesPullingTime="2.737870892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:04:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:04:17.529812225 +0000 UTC m=+1528.019940050" watchObservedRunningTime="2026-10-30 19:04:17.530719197 +0000 UTC m=+1528.020847017" Oct 30 19:04:19.738274 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:19.738235 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:20.120874 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:04:20.120785 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:04:20.283818 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:20.283776 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:23.781726 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:23.781689 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:25.274226 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:25.274190 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:26.909642 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:26.909607 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:27.245280 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:27.245132 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:28.425027 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:28.424998 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj" Oct 30 19:04:30.136127 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:30.136088 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:30.373606 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:30.370867 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:30.734611 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:30.732656 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs"] Oct 30 19:04:30.795595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:30.788529 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs" Oct 30 19:04:30.805037 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:30.790947 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:30.881198 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:30.881163 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-7d059d\"" Oct 30 19:04:30.906627 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:30.905236 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w1-7d059d-dockercfg-4hgpr\"" Oct 30 19:04:31.051663 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:31.029609 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs"] Oct 30 19:04:31.052427 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:31.052396 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc0db36f-b2b2-4add-987b-c0425935d5ae-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs\" (UID: \"fc0db36f-b2b2-4add-987b-c0425935d5ae\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs" Oct 30 19:04:31.091595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:31.081935 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kzl4f\" (UniqueName: \"kubernetes.io/projected/fc0db36f-b2b2-4add-987b-c0425935d5ae-kube-api-access-kzl4f\") pod \"payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs\" (UID: \"fc0db36f-b2b2-4add-987b-c0425935d5ae\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs" Oct 30 19:04:31.286181 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:31.285171 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" podUID="ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded" Oct 30 19:04:31.409598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:31.391706 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:04:31.409598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:31.391794 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" podUID="6364da0d-986e-4551-ae12-b4f3f6963799" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.89:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:04:31.433122 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:31.433077 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc0db36f-b2b2-4add-987b-c0425935d5ae-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs\" (UID: \"fc0db36f-b2b2-4add-987b-c0425935d5ae\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs" Oct 30 19:04:31.457109 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:31.457076 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kzl4f\" (UniqueName: \"kubernetes.io/projected/fc0db36f-b2b2-4add-987b-c0425935d5ae-kube-api-access-kzl4f\") pod \"payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs\" (UID: \"fc0db36f-b2b2-4add-987b-c0425935d5ae\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs" Oct 30 19:04:31.457809 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:31.435114 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc0db36f-b2b2-4add-987b-c0425935d5ae-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs\" (UID: \"fc0db36f-b2b2-4add-987b-c0425935d5ae\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs" Oct 30 19:04:31.500182 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:31.500140 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled (Client.Timeout exceeded while awaiting headers)" Oct 30 19:04:31.548923 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:31.548886 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kzl4f\" (UniqueName: \"kubernetes.io/projected/fc0db36f-b2b2-4add-987b-c0425935d5ae-kube-api-access-kzl4f\") pod \"payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs\" (UID: \"fc0db36f-b2b2-4add-987b-c0425935d5ae\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs" Oct 30 19:04:31.627004 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:31.626968 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs" Oct 30 19:04:31.713738 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:31.699621 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded" Oct 30 19:04:31.723979 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:31.723927 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:04:31.736652 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:31.732892 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:04:32.457294 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:32.457263 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs"] Oct 30 19:04:32.918261 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:32.918229 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs" event={"ID":"fc0db36f-b2b2-4add-987b-c0425935d5ae","Type":"ContainerStarted","Data":"2a96f01078b383102416474b188fb923f01f2b34035c5d0bff1ffaca020ce25d"} Oct 30 19:04:32.918441 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:32.918293 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs" event={"ID":"fc0db36f-b2b2-4add-987b-c0425935d5ae","Type":"ContainerStarted","Data":"5b9ff0641ad04ae0fa18a9af5e02a36e5e5b9d48c2b380849e31557d3621dcc6"} Oct 30 19:04:32.918441 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:32.918322 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs" Oct 30 19:04:34.058977 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:34.058868 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:34.259601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:34.242180 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:36.323270 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:36.311627 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:40.932029 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:40.931996 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:41.011845 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:41.011809 5894 generic.go:356] "Generic (PLEG): container finished" podID="45d20e5e-bffc-4bf1-a51a-592f6c395eb4" containerID="d48c568c10c6ba1eb995553d22df2bd8e319c6888d2dd1ef6cb2face1d9c0c26" exitCode=0 Oct 30 19:04:41.012026 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:41.011881 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" event={"ID":"45d20e5e-bffc-4bf1-a51a-592f6c395eb4","Type":"ContainerDied","Data":"d48c568c10c6ba1eb995553d22df2bd8e319c6888d2dd1ef6cb2face1d9c0c26"} Oct 30 19:04:41.012547 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:41.012529 5894 scope.go:122] "RemoveContainer" containerID="d48c568c10c6ba1eb995553d22df2bd8e319c6888d2dd1ef6cb2face1d9c0c26" Oct 30 19:04:41.014079 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:41.014051 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:04:41.014158 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:41.014118 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:04:41.014158 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:41.014148 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:04:41.122883 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:41.122850 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:41.222067 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:41.221974 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs" podStartSLOduration=11.221959679 podStartE2EDuration="11.221959679s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:04:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:04:33.054726835 +0000 UTC m=+1543.544854657" watchObservedRunningTime="2026-10-30 19:04:41.221959679 +0000 UTC m=+1551.712087498" Oct 30 19:04:41.701669 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:41.698538 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:42.021276 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:42.021188 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" event={"ID":"45d20e5e-bffc-4bf1-a51a-592f6c395eb4","Type":"ContainerStarted","Data":"8047d91c24cae85eb5eb794f955ec3b035415172926a052bc583313dd78bc587"} Oct 30 19:04:42.725921 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:42.725886 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:04:42.750599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:42.749741 5894 patch_prober.go:29] interesting pod/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 30 19:04:42.750599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:42.749823 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" podUID="45d20e5e-bffc-4bf1-a51a-592f6c395eb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 30 19:04:43.018264 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:43.018183 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:43.751601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:43.740929 5894 patch_prober.go:29] interesting pod/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 30 19:04:43.751601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:43.741000 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" podUID="45d20e5e-bffc-4bf1-a51a-592f6c395eb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 30 19:04:43.952473 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:43.952444 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs" Oct 30 19:04:44.726435 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:44.726401 5894 patch_prober.go:29] interesting pod/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 30 19:04:44.726665 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:44.726457 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" podUID="45d20e5e-bffc-4bf1-a51a-592f6c395eb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 30 19:04:44.962042 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:44.962008 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:04:45.726304 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:45.726269 5894 patch_prober.go:29] interesting pod/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 30 19:04:45.726508 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:45.726330 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" podUID="45d20e5e-bffc-4bf1-a51a-592f6c395eb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 30 19:04:46.687816 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:46.687782 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:46.726203 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:46.726174 5894 patch_prober.go:29] interesting pod/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" start-of-body= Oct 30 19:04:46.726378 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:46.726223 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" podUID="45d20e5e-bffc-4bf1-a51a-592f6c395eb4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.87:15021/healthz/ready\": dial tcp 10.132.0.87:15021: connect: connection refused" Oct 30 19:04:46.969513 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:46.969389 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:47.175596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:47.175176 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:47.731728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:47.731700 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:04:47.732931 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:47.732907 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:04:50.118913 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:04:50.118877 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:04:51.825674 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:51.825643 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:52.267029 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:52.266987 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:54.241763 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:54.241730 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:56.388177 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:56.388138 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:56.626594 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:56.618857 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:57.336388 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:57.336358 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:58.203194 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:58.203160 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:04:59.034495 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:04:59.034460 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:00.564994 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:00.564964 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:02.663318 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:02.656207 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:04.294132 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:04.294098 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:07.405634 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:07.403234 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:07.662528 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:07.662425 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:07.863194 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:07.863140 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:12.420202 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:12.420148 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:16.430065 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:16.430032 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk"] Oct 30 19:05:16.430518 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:16.430205 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" podUID="cf26fd33-d037-4630-a87d-ee7487a05eb0" containerName="maas-api" containerID="cri-o://089c359b45ba670f9b1f2c7eec1612b5cf89be5a97c7bd71b73375d5255c9d98" gracePeriod=30 Oct 30 19:05:16.580727 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:16.580676 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:17.040445 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.040411 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" Oct 30 19:05:17.165644 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.165535 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf26fd33-d037-4630-a87d-ee7487a05eb0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf26fd33-d037-4630-a87d-ee7487a05eb0-openshift-service-ca\") pod \"cf26fd33-d037-4630-a87d-ee7487a05eb0\" (UID: \"cf26fd33-d037-4630-a87d-ee7487a05eb0\") " Oct 30 19:05:17.165811 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.165790 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf26fd33-d037-4630-a87d-ee7487a05eb0-kube-api-access-hl657\" (UniqueName: \"kubernetes.io/projected/cf26fd33-d037-4630-a87d-ee7487a05eb0-kube-api-access-hl657\") pod \"cf26fd33-d037-4630-a87d-ee7487a05eb0\" (UID: \"cf26fd33-d037-4630-a87d-ee7487a05eb0\") " Oct 30 19:05:17.165860 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.165846 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf26fd33-d037-4630-a87d-ee7487a05eb0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf26fd33-d037-4630-a87d-ee7487a05eb0-maas-api-tls\") pod \"cf26fd33-d037-4630-a87d-ee7487a05eb0\" (UID: \"cf26fd33-d037-4630-a87d-ee7487a05eb0\") " Oct 30 19:05:17.165899 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.165890 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/cf26fd33-d037-4630-a87d-ee7487a05eb0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf26fd33-d037-4630-a87d-ee7487a05eb0-metrics-certs\") pod \"cf26fd33-d037-4630-a87d-ee7487a05eb0\" (UID: \"cf26fd33-d037-4630-a87d-ee7487a05eb0\") " Oct 30 19:05:17.167453 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.167412 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf26fd33-d037-4630-a87d-ee7487a05eb0-openshift-service-ca" pod "cf26fd33-d037-4630-a87d-ee7487a05eb0" (UID: "cf26fd33-d037-4630-a87d-ee7487a05eb0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:05:17.169242 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.169219 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf26fd33-d037-4630-a87d-ee7487a05eb0-metrics-certs" pod "cf26fd33-d037-4630-a87d-ee7487a05eb0" (UID: "cf26fd33-d037-4630-a87d-ee7487a05eb0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:05:17.169609 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.169556 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf26fd33-d037-4630-a87d-ee7487a05eb0-kube-api-access-hl657" pod "cf26fd33-d037-4630-a87d-ee7487a05eb0" (UID: "cf26fd33-d037-4630-a87d-ee7487a05eb0"). InnerVolumeSpecName "kube-api-access-hl657". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:05:17.170186 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.170165 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cf26fd33-d037-4630-a87d-ee7487a05eb0-maas-api-tls" pod "cf26fd33-d037-4630-a87d-ee7487a05eb0" (UID: "cf26fd33-d037-4630-a87d-ee7487a05eb0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:05:17.268288 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.268252 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/cf26fd33-d037-4630-a87d-ee7487a05eb0-maas-api-tls\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:05:17.268288 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.268282 5894 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/cf26fd33-d037-4630-a87d-ee7487a05eb0-metrics-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:05:17.268288 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.268292 5894 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/cf26fd33-d037-4630-a87d-ee7487a05eb0-openshift-service-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:05:17.268518 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.268301 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl657\" (UniqueName: \"kubernetes.io/projected/cf26fd33-d037-4630-a87d-ee7487a05eb0-kube-api-access-hl657\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:05:17.503367 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.503287 5894 generic.go:356] "Generic (PLEG): container finished" podID="cf26fd33-d037-4630-a87d-ee7487a05eb0" containerID="089c359b45ba670f9b1f2c7eec1612b5cf89be5a97c7bd71b73375d5255c9d98" exitCode=0 Oct 30 19:05:17.503367 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.503334 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" Oct 30 19:05:17.503831 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.503368 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" event={"ID":"cf26fd33-d037-4630-a87d-ee7487a05eb0","Type":"ContainerDied","Data":"089c359b45ba670f9b1f2c7eec1612b5cf89be5a97c7bd71b73375d5255c9d98"} Oct 30 19:05:17.503831 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.503424 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk" event={"ID":"cf26fd33-d037-4630-a87d-ee7487a05eb0","Type":"ContainerDied","Data":"afe185dd90493475f99035ead83af9358f8467b9b7f2f717a8f372c3bcac2d57"} Oct 30 19:05:17.503831 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.503453 5894 scope.go:122] "RemoveContainer" containerID="089c359b45ba670f9b1f2c7eec1612b5cf89be5a97c7bd71b73375d5255c9d98" Oct 30 19:05:17.515381 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.515364 5894 scope.go:122] "RemoveContainer" containerID="089c359b45ba670f9b1f2c7eec1612b5cf89be5a97c7bd71b73375d5255c9d98" Oct 30 19:05:17.515670 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:05:17.515654 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"089c359b45ba670f9b1f2c7eec1612b5cf89be5a97c7bd71b73375d5255c9d98\": container with ID starting with 089c359b45ba670f9b1f2c7eec1612b5cf89be5a97c7bd71b73375d5255c9d98 not found: ID does not exist" containerID="089c359b45ba670f9b1f2c7eec1612b5cf89be5a97c7bd71b73375d5255c9d98" Oct 30 19:05:17.515716 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.515679 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"089c359b45ba670f9b1f2c7eec1612b5cf89be5a97c7bd71b73375d5255c9d98"} err="failed to get container status \"089c359b45ba670f9b1f2c7eec1612b5cf89be5a97c7bd71b73375d5255c9d98\": rpc error: code = NotFound desc = could not find container \"089c359b45ba670f9b1f2c7eec1612b5cf89be5a97c7bd71b73375d5255c9d98\": container with ID starting with 089c359b45ba670f9b1f2c7eec1612b5cf89be5a97c7bd71b73375d5255c9d98 not found: ID does not exist" Oct 30 19:05:17.672220 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.672173 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk"] Oct 30 19:05:17.719908 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:17.719878 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-cfaf8925-6f6fdf5f9c-l6nmk"] Oct 30 19:05:18.093733 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:18.093701 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:18.307524 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:18.307490 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:18.488628 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:18.488595 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:18.794512 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:18.794428 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:19.483865 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:19.483830 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:19.651585 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:19.651552 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf26fd33-d037-4630-a87d-ee7487a05eb0" path="/var/lib/kubelet/pods/cf26fd33-d037-4630-a87d-ee7487a05eb0/volumes" Oct 30 19:05:19.969229 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:19.969189 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7"] Oct 30 19:05:19.970809 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:19.970779 5894 cpu_manager.go:414] "containerMap: removing container" podUID="cf26fd33-d037-4630-a87d-ee7487a05eb0" containerName="maas-api" Oct 30 19:05:19.970809 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:19.970805 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf26fd33-d037-4630-a87d-ee7487a05eb0" containerName="maas-api" Oct 30 19:05:19.971211 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:19.971193 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf26fd33-d037-4630-a87d-ee7487a05eb0" containerName="maas-api" Oct 30 19:05:19.976165 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:19.976142 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.046256 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.046212 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-istio-data\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.046564 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.046541 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7488e3ba-340f-4d02-806d-7b22cc544066-istiod-ca-cert\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.046792 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.046775 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-workload-socket\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.046986 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.046968 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-istio-envoy\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.047122 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.047108 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7488e3ba-340f-4d02-806d-7b22cc544066-istio-podinfo\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.047240 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.047226 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7488e3ba-340f-4d02-806d-7b22cc544066-istio-token\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.047400 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.047375 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-credential-socket\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.047536 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.047523 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-workload-certs\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.047758 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.047742 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br4z9\" (UniqueName: \"kubernetes.io/projected/7488e3ba-340f-4d02-806d-7b22cc544066-kube-api-access-br4z9\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.048277 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.048258 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-eb4c249f-openshift-default-dockercfg-gplr8\"" Oct 30 19:05:20.090388 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.090357 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7"] Oct 30 19:05:20.164660 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.151559 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br4z9\" (UniqueName: \"kubernetes.io/projected/7488e3ba-340f-4d02-806d-7b22cc544066-kube-api-access-br4z9\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.387721 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.387634 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-istio-data\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.388616 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:05:20.217286 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:05:20.389731 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.328702 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-br4z9\" (UniqueName: \"kubernetes.io/projected/7488e3ba-340f-4d02-806d-7b22cc544066-kube-api-access-br4z9\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.389922 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.178444 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:20.390102 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.388276 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-istio-data\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.390234 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.388806 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7488e3ba-340f-4d02-806d-7b22cc544066-istiod-ca-cert\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.390483 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.390466 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-workload-socket\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.397223 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.397194 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-istio-envoy\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.397883 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.397865 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7488e3ba-340f-4d02-806d-7b22cc544066-istio-podinfo\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.399600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.389453 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7488e3ba-340f-4d02-806d-7b22cc544066-istiod-ca-cert\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.399600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.398561 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7488e3ba-340f-4d02-806d-7b22cc544066-istio-token\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.399600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.398727 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-credential-socket\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.399600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.398793 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-workload-certs\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.399600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.397042 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-workload-socket\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.399600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.399318 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-workload-certs\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.399600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.399539 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-credential-socket\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.519264 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.519234 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7488e3ba-340f-4d02-806d-7b22cc544066-istio-podinfo\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.520120 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.520094 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-istio-envoy\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.520293 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.520262 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7488e3ba-340f-4d02-806d-7b22cc544066-istio-token\") pod \"e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.587193 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.587140 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:20.606640 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.600917 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:20.856053 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:20.856022 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:21.133465 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:21.133431 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7"] Oct 30 19:05:21.135020 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:05:21.134996 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7488e3ba_340f_4d02_806d_7b22cc544066.slice/crio-dc728dd52a361a98b3289ae2354d2f9d02535928fa667425a4b888592e0ba179 WatchSource:0}: Error finding container dc728dd52a361a98b3289ae2354d2f9d02535928fa667425a4b888592e0ba179: Status 404 returned error can't find the container with id dc728dd52a361a98b3289ae2354d2f9d02535928fa667425a4b888592e0ba179 Oct 30 19:05:21.138209 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:21.138177 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:05:21.138294 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:21.138237 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:05:21.138294 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:21.138268 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:05:21.561674 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:21.561625 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" event={"ID":"7488e3ba-340f-4d02-806d-7b22cc544066","Type":"ContainerStarted","Data":"0a138608266e629c59a2ef70e51c641345f8e090ba71e56bbead6c1c96273aff"} Oct 30 19:05:21.561873 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:21.561697 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" event={"ID":"7488e3ba-340f-4d02-806d-7b22cc544066","Type":"ContainerStarted","Data":"dc728dd52a361a98b3289ae2354d2f9d02535928fa667425a4b888592e0ba179"} Oct 30 19:05:21.601977 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:21.601943 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:21.728812 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:21.728759 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" podStartSLOduration=2.728743469 podStartE2EDuration="2.728743469s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:21.728212569 +0000 UTC m=+1592.218340391" watchObservedRunningTime="2026-10-30 19:05:21.728743469 +0000 UTC m=+1592.218871288" Oct 30 19:05:22.607076 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:22.607048 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:23.189563 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.189515 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97"] Oct 30 19:05:23.199050 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.199019 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" Oct 30 19:05:23.248750 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.248721 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97"] Oct 30 19:05:23.251009 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.250984 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-w1-7d059d-serving-cert\"" Oct 30 19:05:23.299467 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.299437 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbbmd\" (UniqueName: \"kubernetes.io/projected/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-kube-api-access-mbbmd\") pod \"maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97\" (UID: \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" Oct 30 19:05:23.299650 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.299586 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97\" (UID: \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" Oct 30 19:05:23.299650 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.299619 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97\" (UID: \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" Oct 30 19:05:23.299650 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.299645 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97\" (UID: \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" Oct 30 19:05:23.401389 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.401361 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97\" (UID: \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" Oct 30 19:05:23.401609 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.401394 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97\" (UID: \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" Oct 30 19:05:23.401609 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.401419 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97\" (UID: \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" Oct 30 19:05:23.401609 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:05:23.401526 5894 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-serving-cert: secret "maas-api-e2e-shared-a-w1-7d059d-serving-cert" not found Oct 30 19:05:23.401803 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.401624 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbbmd\" (UniqueName: \"kubernetes.io/projected/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-kube-api-access-mbbmd\") pod \"maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97\" (UID: \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" Oct 30 19:05:23.401803 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:05:23.401644 5894 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-maas-api-tls podName:63ba4b70-66a0-4808-9ff1-f413a6cd6a35 nodeName:}" failed. No retries permitted until 2026-10-30 19:05:23.901621368 +0000 UTC m=+1594.391749172 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-maas-api-tls") pod "maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" (UID: "63ba4b70-66a0-4808-9ff1-f413a6cd6a35") : secret "maas-api-e2e-shared-a-w1-7d059d-serving-cert" not found Oct 30 19:05:23.402252 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.402229 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-openshift-service-ca\") pod \"maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97\" (UID: \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" Oct 30 19:05:23.404052 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.404023 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-metrics-certs\") pod \"maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97\" (UID: \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" Oct 30 19:05:23.422535 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.422497 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:23.484696 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.482736 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbbmd\" (UniqueName: \"kubernetes.io/projected/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-kube-api-access-mbbmd\") pod \"maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97\" (UID: \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" Oct 30 19:05:23.584927 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.584889 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:23.585789 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.585769 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:05:23.629914 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.629887 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:23.846789 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.846654 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:23.968183 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.968142 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97\" (UID: \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" Oct 30 19:05:23.970628 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:23.970605 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-maas-api-tls\") pod \"maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97\" (UID: \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" Oct 30 19:05:24.022351 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:24.022323 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:24.113331 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:24.113243 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" Oct 30 19:05:24.214713 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:24.214675 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:24.496387 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:24.496357 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97"] Oct 30 19:05:24.497671 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:05:24.497428 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63ba4b70_66a0_4808_9ff1_f413a6cd6a35.slice/crio-831a131c3364e4ea6149ccb2ae358a8d4b6dd07c150b0b16bfef2d698d6c900a WatchSource:0}: Error finding container 831a131c3364e4ea6149ccb2ae358a8d4b6dd07c150b0b16bfef2d698d6c900a: Status 404 returned error can't find the container with id 831a131c3364e4ea6149ccb2ae358a8d4b6dd07c150b0b16bfef2d698d6c900a Oct 30 19:05:24.592854 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:24.592821 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" event={"ID":"63ba4b70-66a0-4808-9ff1-f413a6cd6a35","Type":"ContainerStarted","Data":"831a131c3364e4ea6149ccb2ae358a8d4b6dd07c150b0b16bfef2d698d6c900a"} Oct 30 19:05:25.625050 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:25.624840 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" event={"ID":"63ba4b70-66a0-4808-9ff1-f413a6cd6a35","Type":"ContainerStarted","Data":"a2bb2a8d7cdc065a5ed8d2c4ce88f6e8b715922021a34ba7d00e37d03e811cc5"} Oct 30 19:05:25.625050 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:25.624928 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" Oct 30 19:05:25.748569 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:25.748477 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" podStartSLOduration=1.920696857 podStartE2EDuration="2.748458628s" totalImagesPullingTime="827.761771ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:05:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:05:25.748164028 +0000 UTC m=+1596.238291852" watchObservedRunningTime="2026-10-30 19:05:25.748458628 +0000 UTC m=+1596.238586450" Oct 30 19:05:27.189737 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:27.189702 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:27.388680 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:27.388645 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:28.408003 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:28.407963 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:30.207998 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:30.207926 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:30.510866 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:30.510774 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:30.684806 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:30.684524 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:31.056670 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:31.056568 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:31.849367 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:31.849294 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:32.056728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:32.056690 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:32.672939 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:32.672910 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" Oct 30 19:05:35.017259 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.017232 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m"] Oct 30 19:05:35.017657 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.017483 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" podUID="450f937c-7398-4133-b141-18a3ae03f31c" containerName="maas-api" containerID="cri-o://9224a733595d4995141ce1babede88af6686d63ab8f0f44683dd270964281945" gracePeriod=30 Oct 30 19:05:35.159057 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.159023 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:35.685548 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.685044 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" Oct 30 19:05:35.820470 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.820385 5894 generic.go:356] "Generic (PLEG): container finished" podID="450f937c-7398-4133-b141-18a3ae03f31c" containerID="9224a733595d4995141ce1babede88af6686d63ab8f0f44683dd270964281945" exitCode=0 Oct 30 19:05:35.820872 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.820845 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" event={"ID":"450f937c-7398-4133-b141-18a3ae03f31c","Type":"ContainerDied","Data":"9224a733595d4995141ce1babede88af6686d63ab8f0f44683dd270964281945"} Oct 30 19:05:35.821053 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.821036 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" event={"ID":"450f937c-7398-4133-b141-18a3ae03f31c","Type":"ContainerDied","Data":"8bd64f7cb2bd971fe9f6cc8eebc32989e45ab9a029519e4b5e0d84b78a17bbc7"} Oct 30 19:05:35.821183 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.821170 5894 scope.go:122] "RemoveContainer" containerID="9224a733595d4995141ce1babede88af6686d63ab8f0f44683dd270964281945" Oct 30 19:05:35.821435 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.821418 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m" Oct 30 19:05:35.843874 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.843767 5894 scope.go:122] "RemoveContainer" containerID="9224a733595d4995141ce1babede88af6686d63ab8f0f44683dd270964281945" Oct 30 19:05:35.844370 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:05:35.844297 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9224a733595d4995141ce1babede88af6686d63ab8f0f44683dd270964281945\": container with ID starting with 9224a733595d4995141ce1babede88af6686d63ab8f0f44683dd270964281945 not found: ID does not exist" containerID="9224a733595d4995141ce1babede88af6686d63ab8f0f44683dd270964281945" Oct 30 19:05:35.844370 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.844338 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9224a733595d4995141ce1babede88af6686d63ab8f0f44683dd270964281945"} err="failed to get container status \"9224a733595d4995141ce1babede88af6686d63ab8f0f44683dd270964281945\": rpc error: code = NotFound desc = could not find container \"9224a733595d4995141ce1babede88af6686d63ab8f0f44683dd270964281945\": container with ID starting with 9224a733595d4995141ce1babede88af6686d63ab8f0f44683dd270964281945 not found: ID does not exist" Oct 30 19:05:35.869303 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.869277 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/450f937c-7398-4133-b141-18a3ae03f31c-kube-api-access-rqv68\" (UniqueName: \"kubernetes.io/projected/450f937c-7398-4133-b141-18a3ae03f31c-kube-api-access-rqv68\") pod \"450f937c-7398-4133-b141-18a3ae03f31c\" (UID: \"450f937c-7398-4133-b141-18a3ae03f31c\") " Oct 30 19:05:35.869567 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.869545 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/450f937c-7398-4133-b141-18a3ae03f31c-metrics-certs\" (UniqueName: \"kubernetes.io/secret/450f937c-7398-4133-b141-18a3ae03f31c-metrics-certs\") pod \"450f937c-7398-4133-b141-18a3ae03f31c\" (UID: \"450f937c-7398-4133-b141-18a3ae03f31c\") " Oct 30 19:05:35.869957 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.869937 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/450f937c-7398-4133-b141-18a3ae03f31c-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/450f937c-7398-4133-b141-18a3ae03f31c-maas-api-tls\") pod \"450f937c-7398-4133-b141-18a3ae03f31c\" (UID: \"450f937c-7398-4133-b141-18a3ae03f31c\") " Oct 30 19:05:35.870124 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.870108 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/450f937c-7398-4133-b141-18a3ae03f31c-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/450f937c-7398-4133-b141-18a3ae03f31c-openshift-service-ca\") pod \"450f937c-7398-4133-b141-18a3ae03f31c\" (UID: \"450f937c-7398-4133-b141-18a3ae03f31c\") " Oct 30 19:05:35.870967 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.870945 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/450f937c-7398-4133-b141-18a3ae03f31c-openshift-service-ca" pod "450f937c-7398-4133-b141-18a3ae03f31c" (UID: "450f937c-7398-4133-b141-18a3ae03f31c"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:05:35.873122 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.872855 5894 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/450f937c-7398-4133-b141-18a3ae03f31c-openshift-service-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:05:35.875626 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.875078 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/450f937c-7398-4133-b141-18a3ae03f31c-kube-api-access-rqv68" pod "450f937c-7398-4133-b141-18a3ae03f31c" (UID: "450f937c-7398-4133-b141-18a3ae03f31c"). InnerVolumeSpecName "kube-api-access-rqv68". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:05:35.876274 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.875995 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/450f937c-7398-4133-b141-18a3ae03f31c-maas-api-tls" pod "450f937c-7398-4133-b141-18a3ae03f31c" (UID: "450f937c-7398-4133-b141-18a3ae03f31c"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:05:35.876274 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.876136 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/450f937c-7398-4133-b141-18a3ae03f31c-metrics-certs" pod "450f937c-7398-4133-b141-18a3ae03f31c" (UID: "450f937c-7398-4133-b141-18a3ae03f31c"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:05:35.974285 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.974251 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/450f937c-7398-4133-b141-18a3ae03f31c-maas-api-tls\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:05:35.974285 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.974288 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rqv68\" (UniqueName: \"kubernetes.io/projected/450f937c-7398-4133-b141-18a3ae03f31c-kube-api-access-rqv68\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:05:35.974464 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:35.974305 5894 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/450f937c-7398-4133-b141-18a3ae03f31c-metrics-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:05:36.270771 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:36.270732 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m"] Oct 30 19:05:36.335601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:36.322035 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-4b66bd06-67bdffb95d-q8q7m"] Oct 30 19:05:37.030554 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:37.030517 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:37.296148 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:37.292045 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:37.514014 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:37.513984 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:37.656035 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:37.655999 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="450f937c-7398-4133-b141-18a3ae03f31c" path="/var/lib/kubelet/pods/450f937c-7398-4133-b141-18a3ae03f31c/volumes" Oct 30 19:05:37.757196 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:37.757165 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz"] Oct 30 19:05:37.757426 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:37.757401 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" podUID="45d20e5e-bffc-4bf1-a51a-592f6c395eb4" containerName="istio-proxy" containerID="cri-o://8047d91c24cae85eb5eb794f955ec3b035415172926a052bc583313dd78bc587" gracePeriod=30 Oct 30 19:05:37.878595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:37.873957 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:38.701235 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:38.701198 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:38.886769 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:38.886730 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:39.662183 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:39.662150 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:42.612250 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:42.612203 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:42.950900 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:42.950757 5894 generic.go:356] "Generic (PLEG): container finished" podID="45d20e5e-bffc-4bf1-a51a-592f6c395eb4" containerID="8047d91c24cae85eb5eb794f955ec3b035415172926a052bc583313dd78bc587" exitCode=0 Oct 30 19:05:42.950900 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:42.950791 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" event={"ID":"45d20e5e-bffc-4bf1-a51a-592f6c395eb4","Type":"ContainerDied","Data":"8047d91c24cae85eb5eb794f955ec3b035415172926a052bc583313dd78bc587"} Oct 30 19:05:42.950900 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:42.950875 5894 scope.go:122] "RemoveContainer" containerID="d48c568c10c6ba1eb995553d22df2bd8e319c6888d2dd1ef6cb2face1d9c0c26" Oct 30 19:05:43.044972 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.044948 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:05:43.157495 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.157468 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-podinfo\") pod \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " Oct 30 19:05:43.157700 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.157524 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-envoy\") pod \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " Oct 30 19:05:43.157700 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.157552 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-data\") pod \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " Oct 30 19:05:43.157814 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.157698 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-credential-socket\") pod \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " Oct 30 19:05:43.157814 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.157738 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-workload-socket\") pod \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " Oct 30 19:05:43.157917 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.157834 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istiod-ca-cert\") pod \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " Oct 30 19:05:43.157973 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.157926 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-kube-api-access-cn9lj\" (UniqueName: \"kubernetes.io/projected/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-kube-api-access-cn9lj\") pod \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " Oct 30 19:05:43.157973 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.157936 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-data" pod "45d20e5e-bffc-4bf1-a51a-592f6c395eb4" (UID: "45d20e5e-bffc-4bf1-a51a-592f6c395eb4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:05:43.157973 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.157961 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-token\" (UniqueName: \"kubernetes.io/projected/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-token\") pod \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " Oct 30 19:05:43.158113 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.157955 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-credential-socket" pod "45d20e5e-bffc-4bf1-a51a-592f6c395eb4" (UID: "45d20e5e-bffc-4bf1-a51a-592f6c395eb4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:05:43.158113 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.158047 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-workload-socket" pod "45d20e5e-bffc-4bf1-a51a-592f6c395eb4" (UID: "45d20e5e-bffc-4bf1-a51a-592f6c395eb4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:05:43.158211 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.158136 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-workload-certs\") pod \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\" (UID: \"45d20e5e-bffc-4bf1-a51a-592f6c395eb4\") " Oct 30 19:05:43.158373 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.158355 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-workload-certs" pod "45d20e5e-bffc-4bf1-a51a-592f6c395eb4" (UID: "45d20e5e-bffc-4bf1-a51a-592f6c395eb4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:05:43.158588 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.158552 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istiod-ca-cert" pod "45d20e5e-bffc-4bf1-a51a-592f6c395eb4" (UID: "45d20e5e-bffc-4bf1-a51a-592f6c395eb4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:05:43.158856 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.158842 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:05:43.158905 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.158859 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:05:43.158905 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.158869 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:05:43.158905 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.158878 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:05:43.158905 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.158886 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:05:43.159961 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.159936 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-envoy" pod "45d20e5e-bffc-4bf1-a51a-592f6c395eb4" (UID: "45d20e5e-bffc-4bf1-a51a-592f6c395eb4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:05:43.160237 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.160210 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-podinfo" pod "45d20e5e-bffc-4bf1-a51a-592f6c395eb4" (UID: "45d20e5e-bffc-4bf1-a51a-592f6c395eb4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:05:43.160317 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.160260 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-token" pod "45d20e5e-bffc-4bf1-a51a-592f6c395eb4" (UID: "45d20e5e-bffc-4bf1-a51a-592f6c395eb4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:05:43.160475 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.160453 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-kube-api-access-cn9lj" pod "45d20e5e-bffc-4bf1-a51a-592f6c395eb4" (UID: "45d20e5e-bffc-4bf1-a51a-592f6c395eb4"). InnerVolumeSpecName "kube-api-access-cn9lj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:05:43.260175 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.260142 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:05:43.260175 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.260169 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:05:43.260373 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.260184 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cn9lj\" (UniqueName: \"kubernetes.io/projected/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-kube-api-access-cn9lj\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:05:43.260373 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.260198 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/45d20e5e-bffc-4bf1-a51a-592f6c395eb4-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:05:43.959121 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.959031 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" event={"ID":"45d20e5e-bffc-4bf1-a51a-592f6c395eb4","Type":"ContainerDied","Data":"27e6b4618115ada0189901f481e8e9308ce94cf27f64008fe1d04e200120d387"} Oct 30 19:05:43.959121 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.959054 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz" Oct 30 19:05:43.959121 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:43.959087 5894 scope.go:122] "RemoveContainer" containerID="8047d91c24cae85eb5eb794f955ec3b035415172926a052bc583313dd78bc587" Oct 30 19:05:44.175610 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:44.175538 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz"] Oct 30 19:05:44.178642 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:44.178619 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-4b66bd06-openshift-default-68dbd7684f-8c4lz"] Oct 30 19:05:44.750007 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:44.749957 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:45.651960 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:45.651929 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="45d20e5e-bffc-4bf1-a51a-592f6c395eb4" path="/var/lib/kubelet/pods/45d20e5e-bffc-4bf1-a51a-592f6c395eb4/volumes" Oct 30 19:05:47.824471 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:47.824436 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:50.119424 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:05:50.119379 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:05:51.194001 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:51.193969 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:52.684318 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:52.684264 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:05:57.477513 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:05:57.477470 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:02.262656 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:02.257641 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:06.073162 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:06.073122 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:07.438477 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:07.438431 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:11.573441 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:11.573403 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:12.058017 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.057985 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:12.529082 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.529042 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:12.540081 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.540053 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw"] Oct 30 19:06:12.541065 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.541045 5894 cpu_manager.go:414] "containerMap: removing container" podUID="450f937c-7398-4133-b141-18a3ae03f31c" containerName="maas-api" Oct 30 19:06:12.541120 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.541069 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="450f937c-7398-4133-b141-18a3ae03f31c" containerName="maas-api" Oct 30 19:06:12.541120 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.541105 5894 cpu_manager.go:414] "containerMap: removing container" podUID="45d20e5e-bffc-4bf1-a51a-592f6c395eb4" containerName="istio-proxy" Oct 30 19:06:12.541120 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.541113 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="45d20e5e-bffc-4bf1-a51a-592f6c395eb4" containerName="istio-proxy" Oct 30 19:06:12.541219 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.541129 5894 cpu_manager.go:414] "containerMap: removing container" podUID="45d20e5e-bffc-4bf1-a51a-592f6c395eb4" containerName="istio-proxy" Oct 30 19:06:12.541219 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.541138 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="45d20e5e-bffc-4bf1-a51a-592f6c395eb4" containerName="istio-proxy" Oct 30 19:06:12.541327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.541315 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="450f937c-7398-4133-b141-18a3ae03f31c" containerName="maas-api" Oct 30 19:06:12.541361 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.541335 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="45d20e5e-bffc-4bf1-a51a-592f6c395eb4" containerName="istio-proxy" Oct 30 19:06:12.541361 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.541347 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="45d20e5e-bffc-4bf1-a51a-592f6c395eb4" containerName="istio-proxy" Oct 30 19:06:12.545458 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.545439 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.612708 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.612670 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-envoy\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.613251 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.613221 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-data\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.613383 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.613269 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-token\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.613383 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.613324 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-workload-socket\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.613488 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.613439 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-podinfo\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.613488 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.613463 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8db11496-7f7c-49f1-ba40-08248a5b2db2-istiod-ca-cert\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.613663 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.613490 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-workload-certs\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.613663 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.613607 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79fpd\" (UniqueName: \"kubernetes.io/projected/8db11496-7f7c-49f1-ba40-08248a5b2db2-kube-api-access-79fpd\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.613776 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.613713 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-credential-socket\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.640695 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.640658 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw"] Oct 30 19:06:12.663428 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.663394 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w3-732a3f-openshift-default-dockercfg-b9zk4\"" Oct 30 19:06:12.746598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.719638 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-envoy\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.746905 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.746877 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-data\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.747058 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.747038 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-token\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.747218 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.747203 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-workload-socket\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.747443 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.747427 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-podinfo\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.747601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.747568 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8db11496-7f7c-49f1-ba40-08248a5b2db2-istiod-ca-cert\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.747746 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.747730 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-workload-certs\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.747951 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.747935 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79fpd\" (UniqueName: \"kubernetes.io/projected/8db11496-7f7c-49f1-ba40-08248a5b2db2-kube-api-access-79fpd\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.748168 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.748149 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-credential-socket\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.779644 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.749080 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-data\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.806596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.753830 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-workload-socket\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.806596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.775377 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8db11496-7f7c-49f1-ba40-08248a5b2db2-istiod-ca-cert\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.806596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.783739 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-workload-certs\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.806596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.784039 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-credential-socket\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.844011 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.838080 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-podinfo\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.865381 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.861940 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-envoy\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.884036 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.883974 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:12.912954 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.912910 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-79fpd\" (UniqueName: \"kubernetes.io/projected/8db11496-7f7c-49f1-ba40-08248a5b2db2-kube-api-access-79fpd\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:12.984638 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:12.983831 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-token\") pod \"e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:13.117409 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:13.117327 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:13.168599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:13.162896 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:13.696813 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:13.696785 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw"] Oct 30 19:06:13.698683 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:06:13.698652 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8db11496_7f7c_49f1_ba40_08248a5b2db2.slice/crio-20cd7f7b02502520a9e2c7a09760f45ad8fb1ed8d0ab86b6bd70b92fb7fce402 WatchSource:0}: Error finding container 20cd7f7b02502520a9e2c7a09760f45ad8fb1ed8d0ab86b6bd70b92fb7fce402: Status 404 returned error can't find the container with id 20cd7f7b02502520a9e2c7a09760f45ad8fb1ed8d0ab86b6bd70b92fb7fce402 Oct 30 19:06:13.701614 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:13.701563 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:06:13.701698 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:13.701659 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:06:13.701739 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:13.701700 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:06:13.776157 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:13.776129 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:14.160910 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:14.160860 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" event={"ID":"8db11496-7f7c-49f1-ba40-08248a5b2db2","Type":"ContainerStarted","Data":"55a641a7e5e99592f04a6527fa9239ffdc56443ae313778feaa06dd679f9ef81"} Oct 30 19:06:14.161113 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:14.160922 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" event={"ID":"8db11496-7f7c-49f1-ba40-08248a5b2db2","Type":"ContainerStarted","Data":"20cd7f7b02502520a9e2c7a09760f45ad8fb1ed8d0ab86b6bd70b92fb7fce402"} Oct 30 19:06:14.164005 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:14.163977 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:14.171105 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:14.171079 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:14.287071 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:14.287012 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" podStartSLOduration=2.286993199 podStartE2EDuration="2.286993199s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:14.283284579 +0000 UTC m=+1644.773412399" watchObservedRunningTime="2026-10-30 19:06:14.286993199 +0000 UTC m=+1644.777121024" Oct 30 19:06:15.168312 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:15.168286 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:15.169310 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:15.169289 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:06:17.314395 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:17.314361 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:17.600870 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:17.600784 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:17.899025 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:17.898988 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:18.136945 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:18.136910 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:18.413923 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:18.413895 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:20.121702 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:06:20.121480 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:06:21.140506 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:21.129623 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:21.345300 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:21.345206 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:23.165079 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:23.165041 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:24.581457 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:24.581396 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:25.125851 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.125800 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:25.372600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.365524 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh"] Oct 30 19:06:25.391666 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.391630 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.461248 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.461217 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-4501c8d4-openshift-default-dockercfg-r7jlc\"" Oct 30 19:06:25.462975 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.462841 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-credential-socket\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.463247 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.463229 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-envoy\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.463430 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.463398 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bdpsg\" (UniqueName: \"kubernetes.io/projected/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-kube-api-access-bdpsg\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.463658 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.463627 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-podinfo\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.463794 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.463777 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istiod-ca-cert\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.463958 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.463943 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-workload-socket\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.464069 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.464054 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-token\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.464174 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.464160 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-workload-certs\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.464327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.464310 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-data\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.464889 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.464871 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh"] Oct 30 19:06:25.579538 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.579217 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:25.596156 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.596119 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-podinfo\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.596744 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.596717 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istiod-ca-cert\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.596937 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.596920 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-workload-socket\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.597063 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.597049 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-token\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.597175 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.597160 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-workload-certs\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.597336 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.597318 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-data\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.597618 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.597599 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-credential-socket\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.597877 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.597858 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-envoy\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.598065 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.598049 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bdpsg\" (UniqueName: \"kubernetes.io/projected/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-kube-api-access-bdpsg\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.603395 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.603369 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-workload-certs\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.605962 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.604237 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istiod-ca-cert\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.617189 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.617157 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-workload-socket\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.619667 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.617958 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-credential-socket\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.619851 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.618192 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-data\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.659202 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.635701 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-envoy\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.724494 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.724425 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-podinfo\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.782675 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.781467 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-token\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:25.782675 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:25.782334 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bdpsg\" (UniqueName: \"kubernetes.io/projected/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-kube-api-access-bdpsg\") pod \"e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:26.085161 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:26.084459 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:26.203321 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:26.203235 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:26.596909 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:26.596879 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh"] Oct 30 19:06:26.597992 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:06:26.597967 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef4b2f6f_0e65_4dd4_b028_c50314c9e7e4.slice/crio-12f4d63df275f0685076f21b06965dd923e77427452e9ed94362eb4376cc9ac6 WatchSource:0}: Error finding container 12f4d63df275f0685076f21b06965dd923e77427452e9ed94362eb4376cc9ac6: Status 404 returned error can't find the container with id 12f4d63df275f0685076f21b06965dd923e77427452e9ed94362eb4376cc9ac6 Oct 30 19:06:26.601474 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:26.601432 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:06:26.601585 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:26.601531 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:06:26.601636 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:26.601614 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:06:27.382583 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:27.382536 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" event={"ID":"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4","Type":"ContainerStarted","Data":"2ef858e03d4bb08b4e636b00ce8def883e42d62a73c340fbcd1ecd0236b88406"} Oct 30 19:06:27.382756 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:27.382609 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" event={"ID":"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4","Type":"ContainerStarted","Data":"12f4d63df275f0685076f21b06965dd923e77427452e9ed94362eb4376cc9ac6"} Oct 30 19:06:27.756671 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:27.745986 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" podStartSLOduration=2.745955016 podStartE2EDuration="2.745955016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:27.563129227 +0000 UTC m=+1658.053257044" watchObservedRunningTime="2026-10-30 19:06:27.745955016 +0000 UTC m=+1658.236082835" Oct 30 19:06:28.088602 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:28.088493 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:28.097509 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:28.097474 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:28.389701 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:28.389626 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:28.390603 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:28.390558 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:06:28.588338 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:28.588294 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:29.417239 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:29.417203 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:29.698167 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:29.698079 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:30.154211 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:30.154177 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:30.795608 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:30.794006 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:31.034896 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.033039 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:31.249295 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.249264 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2"] Oct 30 19:06:31.253816 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.253798 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.306858 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.306827 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-87ed12-openshift-default-dockercfg-f4qfc\"" Oct 30 19:06:31.316150 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.315541 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-envoy\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.316410 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.316387 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-podinfo\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.316554 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.316539 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dx9zb\" (UniqueName: \"kubernetes.io/projected/2a4529e2-c317-4d6d-916f-b77baf48009a-kube-api-access-dx9zb\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.316685 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.316671 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-token\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.317037 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.317009 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-data\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.317167 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.317151 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-workload-socket\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.317307 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.317294 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-workload-certs\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.317421 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.317409 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-credential-socket\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.317543 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.317531 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a4529e2-c317-4d6d-916f-b77baf48009a-istiod-ca-cert\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.362567 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.362536 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2"] Oct 30 19:06:31.471593 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.462247 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-data\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.471593 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.462324 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-workload-socket\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.471593 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.462417 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-workload-certs\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.471593 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.463361 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-workload-certs\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.471593 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.463607 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-workload-socket\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.471593 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.463740 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-credential-socket\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.471593 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.464114 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-credential-socket\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.471593 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.464201 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a4529e2-c317-4d6d-916f-b77baf48009a-istiod-ca-cert\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.471593 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.464401 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-envoy\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.471593 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.464479 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-podinfo\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.471593 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.464531 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dx9zb\" (UniqueName: \"kubernetes.io/projected/2a4529e2-c317-4d6d-916f-b77baf48009a-kube-api-access-dx9zb\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.471593 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.464600 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-token\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.471593 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.465628 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-data\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.471593 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.466322 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a4529e2-c317-4d6d-916f-b77baf48009a-istiod-ca-cert\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.545703 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.496348 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:31.545703 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.499967 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-envoy\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.545703 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.529507 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-podinfo\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.559140 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.559100 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-token\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.561932 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.561901 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dx9zb\" (UniqueName: \"kubernetes.io/projected/2a4529e2-c317-4d6d-916f-b77baf48009a-kube-api-access-dx9zb\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.581869 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.581836 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:31.710666 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:31.704584 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:32.038426 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:32.038366 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:32.195604 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:32.186886 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2"] Oct 30 19:06:32.207911 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:06:32.207824 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a4529e2_c317_4d6d_916f_b77baf48009a.slice/crio-054c76e07ad03af49b1365967b1a930f225242319ffe80bc6cd5483d0ff89d48 WatchSource:0}: Error finding container 054c76e07ad03af49b1365967b1a930f225242319ffe80bc6cd5483d0ff89d48: Status 404 returned error can't find the container with id 054c76e07ad03af49b1365967b1a930f225242319ffe80bc6cd5483d0ff89d48 Oct 30 19:06:32.212521 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:32.211136 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:06:32.212521 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:32.211232 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:06:32.212521 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:32.211273 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:06:32.525218 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:32.525178 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" event={"ID":"2a4529e2-c317-4d6d-916f-b77baf48009a","Type":"ContainerStarted","Data":"6f68a8b0b7791d38ab0ecdac757da4175e5501417c6265b6ef12dd5c03881a00"} Oct 30 19:06:32.525408 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:32.525243 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" event={"ID":"2a4529e2-c317-4d6d-916f-b77baf48009a","Type":"ContainerStarted","Data":"054c76e07ad03af49b1365967b1a930f225242319ffe80bc6cd5483d0ff89d48"} Oct 30 19:06:32.583705 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:32.583617 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:32.649966 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:32.649917 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" podStartSLOduration=1.649901663 podStartE2EDuration="1.649901663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:06:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:06:32.647173314 +0000 UTC m=+1663.137301133" watchObservedRunningTime="2026-10-30 19:06:32.649901663 +0000 UTC m=+1663.140029549" Oct 30 19:06:33.589808 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:33.589778 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:34.100770 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:34.100735 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:34.443464 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:34.443425 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:34.578181 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:34.578150 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:34.643596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:34.633886 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:06:34.731695 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:34.721132 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:35.394201 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:35.394165 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:35.609227 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:35.609193 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:35.885296 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:35.885210 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:36.100162 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:36.100131 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:39.010130 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:39.010092 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:39.217314 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:39.217277 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:40.377447 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:40.377409 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:41.564880 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:41.564835 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:41.748009 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:41.747973 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:45.201351 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:45.201317 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:50.092795 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:50.092756 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:50.213896 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:06:50.213855 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:06:54.767239 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:54.767200 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:06:59.437710 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:06:59.437668 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:04.307597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:04.306789 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:06.614696 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:06.614665 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254"] Oct 30 19:07:06.615313 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:06.615283 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" podUID="e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" containerName="istio-proxy" containerID="cri-o://3a1ec1750c46d2c6084e84213c21ee7084d45aab104ce63050bb5f7e99ea6adb" gracePeriod=30 Oct 30 19:07:06.664929 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:06.664894 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv"] Oct 30 19:07:06.665301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:06.665273 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" podUID="a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" containerName="istio-proxy" containerID="cri-o://0eb55dd835c159c05b7055aa9a1a603889999cc57a1589f5ff4f7df9221f0653" gracePeriod=30 Oct 30 19:07:06.668311 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:06.668287 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh"] Oct 30 19:07:06.668488 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:06.668468 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" podUID="d1d373aa-3cfa-4f6a-9746-889e0d90cde2" containerName="istio-proxy" containerID="cri-o://f9e63c22c869e759a80c0049dae21694907a8535ee8c63eb75a927ac6e22f99d" gracePeriod=30 Oct 30 19:07:06.716109 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:06.716080 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq"] Oct 30 19:07:06.716482 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:06.716442 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" podUID="43bf10dc-7107-4803-a386-c98e0a9c3bc2" containerName="istio-proxy" containerID="cri-o://2fd54502511abc944d1a825d40a66d16f9cc22faf9c1a3f4e9cb6070e5a94adc" gracePeriod=30 Oct 30 19:07:06.848596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:06.847179 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:09.046599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:09.043313 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:10.216622 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:10.216545 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:07:10.216622 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:10.216606 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" podUID="d1d373aa-3cfa-4f6a-9746-889e0d90cde2" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:07:11.346940 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:11.346909 5894 patch_prober.go:29] interesting pod/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:07:11.347323 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:11.346959 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" podUID="a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:07:12.007529 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.007502 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:07:12.106837 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.105269 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-podinfo\") pod \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " Oct 30 19:07:12.106837 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.105349 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istiod-ca-cert\") pod \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " Oct 30 19:07:12.106837 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.105432 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-workload-socket\") pod \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " Oct 30 19:07:12.106837 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.105530 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-credential-socket\") pod \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " Oct 30 19:07:12.106837 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.105566 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-envoy\") pod \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " Oct 30 19:07:12.106837 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.105623 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-kube-api-access-bqtmq\" (UniqueName: \"kubernetes.io/projected/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-kube-api-access-bqtmq\") pod \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " Oct 30 19:07:12.106837 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.105663 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-token\" (UniqueName: \"kubernetes.io/projected/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-token\") pod \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " Oct 30 19:07:12.106837 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.105854 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-workload-certs\") pod \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " Oct 30 19:07:12.106837 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.105914 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-data\") pod \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\" (UID: \"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb\") " Oct 30 19:07:12.107957 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.107621 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-credential-socket" pod "e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" (UID: "e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.108629 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.108592 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-workload-socket" pod "e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" (UID: "e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.108910 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.108854 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-workload-certs" pod "e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" (UID: "e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.108910 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.108871 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-data" pod "e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" (UID: "e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.110103 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.109453 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istiod-ca-cert" pod "e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" (UID: "e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:07:12.114838 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.113822 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-kube-api-access-bqtmq" pod "e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" (UID: "e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb"). InnerVolumeSpecName "kube-api-access-bqtmq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:07:12.114838 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.114105 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-token" pod "e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" (UID: "e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:07:12.116268 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.116230 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 19:07:12.118916 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.118835 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 19:07:12.120934 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.120885 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-podinfo" pod "e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" (UID: "e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:07:12.122552 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.122519 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-envoy" pod "e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" (UID: "e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.145959 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.145932 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:07:12.175260 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.175122 5894 generic.go:356] "Generic (PLEG): container finished" podID="e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" containerID="3a1ec1750c46d2c6084e84213c21ee7084d45aab104ce63050bb5f7e99ea6adb" exitCode=0 Oct 30 19:07:12.175260 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.175190 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" Oct 30 19:07:12.175260 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.175205 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" event={"ID":"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb","Type":"ContainerDied","Data":"3a1ec1750c46d2c6084e84213c21ee7084d45aab104ce63050bb5f7e99ea6adb"} Oct 30 19:07:12.175553 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.175287 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254" event={"ID":"e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb","Type":"ContainerDied","Data":"c766f69c6c0942fc293744b7fcae9549841e93e560f096fbdd9e71887a966e12"} Oct 30 19:07:12.175553 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.175341 5894 scope.go:122] "RemoveContainer" containerID="3a1ec1750c46d2c6084e84213c21ee7084d45aab104ce63050bb5f7e99ea6adb" Oct 30 19:07:12.180662 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.180632 5894 generic.go:356] "Generic (PLEG): container finished" podID="a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" containerID="0eb55dd835c159c05b7055aa9a1a603889999cc57a1589f5ff4f7df9221f0653" exitCode=0 Oct 30 19:07:12.180789 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.180695 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" Oct 30 19:07:12.180789 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.180752 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" event={"ID":"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3","Type":"ContainerDied","Data":"0eb55dd835c159c05b7055aa9a1a603889999cc57a1589f5ff4f7df9221f0653"} Oct 30 19:07:12.180910 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.180797 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv" event={"ID":"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3","Type":"ContainerDied","Data":"d0d9c5840d0bcf7d97fff18781b48c6d5030ca0998ad72a88e69130a2ca390d3"} Oct 30 19:07:12.182330 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.182303 5894 generic.go:356] "Generic (PLEG): container finished" podID="d1d373aa-3cfa-4f6a-9746-889e0d90cde2" containerID="f9e63c22c869e759a80c0049dae21694907a8535ee8c63eb75a927ac6e22f99d" exitCode=0 Oct 30 19:07:12.183157 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.182620 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" Oct 30 19:07:12.184244 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.184221 5894 generic.go:356] "Generic (PLEG): container finished" podID="43bf10dc-7107-4803-a386-c98e0a9c3bc2" containerID="2fd54502511abc944d1a825d40a66d16f9cc22faf9c1a3f4e9cb6070e5a94adc" exitCode=0 Oct 30 19:07:12.184501 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.184357 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" Oct 30 19:07:12.184770 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.184745 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" event={"ID":"d1d373aa-3cfa-4f6a-9746-889e0d90cde2","Type":"ContainerDied","Data":"f9e63c22c869e759a80c0049dae21694907a8535ee8c63eb75a927ac6e22f99d"} Oct 30 19:07:12.184852 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.184786 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh" event={"ID":"d1d373aa-3cfa-4f6a-9746-889e0d90cde2","Type":"ContainerDied","Data":"6982fdd8e093acc11f4c91714eb8008fa88fefe1fd71f951aeb13f655eb74929"} Oct 30 19:07:12.184852 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.184807 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" event={"ID":"43bf10dc-7107-4803-a386-c98e0a9c3bc2","Type":"ContainerDied","Data":"2fd54502511abc944d1a825d40a66d16f9cc22faf9c1a3f4e9cb6070e5a94adc"} Oct 30 19:07:12.184852 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.184832 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq" event={"ID":"43bf10dc-7107-4803-a386-c98e0a9c3bc2","Type":"ContainerDied","Data":"8933cddcdd22c429a0fca6afc4df64bb2c5220f05cc6ed8401ad28641899a548"} Oct 30 19:07:12.192817 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.192614 5894 scope.go:122] "RemoveContainer" containerID="3a1ec1750c46d2c6084e84213c21ee7084d45aab104ce63050bb5f7e99ea6adb" Oct 30 19:07:12.192969 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:07:12.192916 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a1ec1750c46d2c6084e84213c21ee7084d45aab104ce63050bb5f7e99ea6adb\": container with ID starting with 3a1ec1750c46d2c6084e84213c21ee7084d45aab104ce63050bb5f7e99ea6adb not found: ID does not exist" containerID="3a1ec1750c46d2c6084e84213c21ee7084d45aab104ce63050bb5f7e99ea6adb" Oct 30 19:07:12.192969 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.192947 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a1ec1750c46d2c6084e84213c21ee7084d45aab104ce63050bb5f7e99ea6adb"} err="failed to get container status \"3a1ec1750c46d2c6084e84213c21ee7084d45aab104ce63050bb5f7e99ea6adb\": rpc error: code = NotFound desc = could not find container \"3a1ec1750c46d2c6084e84213c21ee7084d45aab104ce63050bb5f7e99ea6adb\": container with ID starting with 3a1ec1750c46d2c6084e84213c21ee7084d45aab104ce63050bb5f7e99ea6adb not found: ID does not exist" Oct 30 19:07:12.193105 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.192972 5894 scope.go:122] "RemoveContainer" containerID="0eb55dd835c159c05b7055aa9a1a603889999cc57a1589f5ff4f7df9221f0653" Oct 30 19:07:12.207727 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.207655 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-credential-socket\") pod \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " Oct 30 19:07:12.207868 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.207739 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-kube-api-access-mn58g\" (UniqueName: \"kubernetes.io/projected/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-kube-api-access-mn58g\") pod \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " Oct 30 19:07:12.207868 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.207775 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-workload-certs\") pod \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " Oct 30 19:07:12.207868 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.207813 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-workload-socket\") pod \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " Oct 30 19:07:12.207868 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.207863 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istiod-ca-cert\") pod \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " Oct 30 19:07:12.208088 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.207944 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-envoy\") pod \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " Oct 30 19:07:12.208088 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.208026 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-token\" (UniqueName: \"kubernetes.io/projected/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-token\") pod \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " Oct 30 19:07:12.208195 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.208088 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-kube-api-access-nc2hh\" (UniqueName: \"kubernetes.io/projected/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-kube-api-access-nc2hh\") pod \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " Oct 30 19:07:12.208195 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.208128 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-workload-socket\") pod \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " Oct 30 19:07:12.208306 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.208237 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-credential-socket\") pod \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " Oct 30 19:07:12.208367 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.208336 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-token\" (UniqueName: \"kubernetes.io/projected/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-token\") pod \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " Oct 30 19:07:12.208420 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.208383 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-envoy\") pod \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " Oct 30 19:07:12.208476 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.208415 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-workload-certs\") pod \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " Oct 30 19:07:12.208476 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.208469 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-podinfo\") pod \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " Oct 30 19:07:12.208963 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.208501 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-data\") pod \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " Oct 30 19:07:12.208963 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.208543 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-podinfo\") pod \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " Oct 30 19:07:12.208963 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.208617 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istiod-ca-cert\") pod \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\" (UID: \"d1d373aa-3cfa-4f6a-9746-889e0d90cde2\") " Oct 30 19:07:12.208963 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.208650 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-data\") pod \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\" (UID: \"a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3\") " Oct 30 19:07:12.209416 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.209394 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.209501 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.209423 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.209501 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.209438 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.209501 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.209451 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.209501 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.209463 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.209501 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.209478 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.209501 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.209491 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.209837 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.209505 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bqtmq\" (UniqueName: \"kubernetes.io/projected/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-kube-api-access-bqtmq\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.209837 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.209519 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.209939 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.209830 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-data" pod "a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" (UID: "a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.210840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.210031 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-credential-socket" pod "a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" (UID: "a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.210840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.210240 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-credential-socket" pod "d1d373aa-3cfa-4f6a-9746-889e0d90cde2" (UID: "d1d373aa-3cfa-4f6a-9746-889e0d90cde2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.210840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.210275 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-workload-socket" pod "a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" (UID: "a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.211265 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.211248 5894 scope.go:122] "RemoveContainer" containerID="0eb55dd835c159c05b7055aa9a1a603889999cc57a1589f5ff4f7df9221f0653" Oct 30 19:07:12.211512 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.211281 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-workload-certs" pod "d1d373aa-3cfa-4f6a-9746-889e0d90cde2" (UID: "d1d373aa-3cfa-4f6a-9746-889e0d90cde2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.211857 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.211294 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-workload-certs" pod "a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" (UID: "a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.211857 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.211662 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-data" pod "d1d373aa-3cfa-4f6a-9746-889e0d90cde2" (UID: "d1d373aa-3cfa-4f6a-9746-889e0d90cde2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.211857 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.211791 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istiod-ca-cert" pod "d1d373aa-3cfa-4f6a-9746-889e0d90cde2" (UID: "d1d373aa-3cfa-4f6a-9746-889e0d90cde2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:07:12.212058 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.211927 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-workload-socket" pod "d1d373aa-3cfa-4f6a-9746-889e0d90cde2" (UID: "d1d373aa-3cfa-4f6a-9746-889e0d90cde2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.212113 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:07:12.212062 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0eb55dd835c159c05b7055aa9a1a603889999cc57a1589f5ff4f7df9221f0653\": container with ID starting with 0eb55dd835c159c05b7055aa9a1a603889999cc57a1589f5ff4f7df9221f0653 not found: ID does not exist" containerID="0eb55dd835c159c05b7055aa9a1a603889999cc57a1589f5ff4f7df9221f0653" Oct 30 19:07:12.212175 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.212102 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0eb55dd835c159c05b7055aa9a1a603889999cc57a1589f5ff4f7df9221f0653"} err="failed to get container status \"0eb55dd835c159c05b7055aa9a1a603889999cc57a1589f5ff4f7df9221f0653\": rpc error: code = NotFound desc = could not find container \"0eb55dd835c159c05b7055aa9a1a603889999cc57a1589f5ff4f7df9221f0653\": container with ID starting with 0eb55dd835c159c05b7055aa9a1a603889999cc57a1589f5ff4f7df9221f0653 not found: ID does not exist" Oct 30 19:07:12.212175 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.212139 5894 scope.go:122] "RemoveContainer" containerID="f9e63c22c869e759a80c0049dae21694907a8535ee8c63eb75a927ac6e22f99d" Oct 30 19:07:12.212283 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.212167 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istiod-ca-cert" pod "a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" (UID: "a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:07:12.214189 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.213967 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-kube-api-access-mn58g" pod "a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" (UID: "a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3"). InnerVolumeSpecName "kube-api-access-mn58g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:07:12.214189 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.214046 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-token" pod "d1d373aa-3cfa-4f6a-9746-889e0d90cde2" (UID: "d1d373aa-3cfa-4f6a-9746-889e0d90cde2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:07:12.216133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.216093 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-envoy" pod "d1d373aa-3cfa-4f6a-9746-889e0d90cde2" (UID: "d1d373aa-3cfa-4f6a-9746-889e0d90cde2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.216952 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.216878 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-podinfo" pod "d1d373aa-3cfa-4f6a-9746-889e0d90cde2" (UID: "d1d373aa-3cfa-4f6a-9746-889e0d90cde2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:07:12.216952 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.216914 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-token" pod "a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" (UID: "a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:07:12.217109 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.217023 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-podinfo" pod "a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" (UID: "a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:07:12.217860 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.217834 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-envoy" pod "a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" (UID: "a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.218337 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.218312 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-kube-api-access-nc2hh" pod "d1d373aa-3cfa-4f6a-9746-889e0d90cde2" (UID: "d1d373aa-3cfa-4f6a-9746-889e0d90cde2"). InnerVolumeSpecName "kube-api-access-nc2hh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:07:12.227677 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.227538 5894 scope.go:122] "RemoveContainer" containerID="f9e63c22c869e759a80c0049dae21694907a8535ee8c63eb75a927ac6e22f99d" Oct 30 19:07:12.228008 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:07:12.227977 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9e63c22c869e759a80c0049dae21694907a8535ee8c63eb75a927ac6e22f99d\": container with ID starting with f9e63c22c869e759a80c0049dae21694907a8535ee8c63eb75a927ac6e22f99d not found: ID does not exist" containerID="f9e63c22c869e759a80c0049dae21694907a8535ee8c63eb75a927ac6e22f99d" Oct 30 19:07:12.228096 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.228021 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9e63c22c869e759a80c0049dae21694907a8535ee8c63eb75a927ac6e22f99d"} err="failed to get container status \"f9e63c22c869e759a80c0049dae21694907a8535ee8c63eb75a927ac6e22f99d\": rpc error: code = NotFound desc = could not find container \"f9e63c22c869e759a80c0049dae21694907a8535ee8c63eb75a927ac6e22f99d\": container with ID starting with f9e63c22c869e759a80c0049dae21694907a8535ee8c63eb75a927ac6e22f99d not found: ID does not exist" Oct 30 19:07:12.228096 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.228046 5894 scope.go:122] "RemoveContainer" containerID="2fd54502511abc944d1a825d40a66d16f9cc22faf9c1a3f4e9cb6070e5a94adc" Oct 30 19:07:12.242487 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.242461 5894 scope.go:122] "RemoveContainer" containerID="2fd54502511abc944d1a825d40a66d16f9cc22faf9c1a3f4e9cb6070e5a94adc" Oct 30 19:07:12.243020 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:07:12.242994 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fd54502511abc944d1a825d40a66d16f9cc22faf9c1a3f4e9cb6070e5a94adc\": container with ID starting with 2fd54502511abc944d1a825d40a66d16f9cc22faf9c1a3f4e9cb6070e5a94adc not found: ID does not exist" containerID="2fd54502511abc944d1a825d40a66d16f9cc22faf9c1a3f4e9cb6070e5a94adc" Oct 30 19:07:12.243126 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.243027 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fd54502511abc944d1a825d40a66d16f9cc22faf9c1a3f4e9cb6070e5a94adc"} err="failed to get container status \"2fd54502511abc944d1a825d40a66d16f9cc22faf9c1a3f4e9cb6070e5a94adc\": rpc error: code = NotFound desc = could not find container \"2fd54502511abc944d1a825d40a66d16f9cc22faf9c1a3f4e9cb6070e5a94adc\": container with ID starting with 2fd54502511abc944d1a825d40a66d16f9cc22faf9c1a3f4e9cb6070e5a94adc not found: ID does not exist" Oct 30 19:07:12.310054 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.310020 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-token\" (UniqueName: \"kubernetes.io/projected/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-token\") pod \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " Oct 30 19:07:12.310232 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.310070 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/43bf10dc-7107-4803-a386-c98e0a9c3bc2-kube-api-access-8bgwn\" (UniqueName: \"kubernetes.io/projected/43bf10dc-7107-4803-a386-c98e0a9c3bc2-kube-api-access-8bgwn\") pod \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " Oct 30 19:07:12.310232 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.310177 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istiod-ca-cert\") pod \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " Oct 30 19:07:12.310368 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.310244 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-podinfo\") pod \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " Oct 30 19:07:12.310368 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.310283 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-credential-socket\") pod \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " Oct 30 19:07:12.310368 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.310347 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-data\") pod \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " Oct 30 19:07:12.310536 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.310386 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-workload-socket\") pod \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " Oct 30 19:07:12.310536 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.310523 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-workload-certs\") pod \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " Oct 30 19:07:12.310685 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.310656 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-credential-socket" pod "43bf10dc-7107-4803-a386-c98e0a9c3bc2" (UID: "43bf10dc-7107-4803-a386-c98e0a9c3bc2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.310985 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.310704 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-envoy\") pod \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\" (UID: \"43bf10dc-7107-4803-a386-c98e0a9c3bc2\") " Oct 30 19:07:12.310985 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.310771 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-workload-socket" pod "43bf10dc-7107-4803-a386-c98e0a9c3bc2" (UID: "43bf10dc-7107-4803-a386-c98e0a9c3bc2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.310985 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.310705 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istiod-ca-cert" pod "43bf10dc-7107-4803-a386-c98e0a9c3bc2" (UID: "43bf10dc-7107-4803-a386-c98e0a9c3bc2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:07:12.310985 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.310827 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-data" pod "43bf10dc-7107-4803-a386-c98e0a9c3bc2" (UID: "43bf10dc-7107-4803-a386-c98e0a9c3bc2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.310985 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.310948 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-workload-certs" pod "43bf10dc-7107-4803-a386-c98e0a9c3bc2" (UID: "43bf10dc-7107-4803-a386-c98e0a9c3bc2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.311504 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311440 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311504 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311467 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mn58g\" (UniqueName: \"kubernetes.io/projected/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-kube-api-access-mn58g\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311504 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311484 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311504 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311497 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311511 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311524 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311536 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311551 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311564 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311596 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nc2hh\" (UniqueName: \"kubernetes.io/projected/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-kube-api-access-nc2hh\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311611 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311624 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311637 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311650 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311663 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311676 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311690 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311704 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311718 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311731 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311744 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311756 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d1d373aa-3cfa-4f6a-9746-889e0d90cde2-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.311798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.311770 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.314868 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.314844 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-token" pod "43bf10dc-7107-4803-a386-c98e0a9c3bc2" (UID: "43bf10dc-7107-4803-a386-c98e0a9c3bc2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:07:12.315117 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.315069 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43bf10dc-7107-4803-a386-c98e0a9c3bc2-kube-api-access-8bgwn" pod "43bf10dc-7107-4803-a386-c98e0a9c3bc2" (UID: "43bf10dc-7107-4803-a386-c98e0a9c3bc2"). InnerVolumeSpecName "kube-api-access-8bgwn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:07:12.319955 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.319933 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-envoy" pod "43bf10dc-7107-4803-a386-c98e0a9c3bc2" (UID: "43bf10dc-7107-4803-a386-c98e0a9c3bc2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:07:12.321242 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.321217 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-podinfo" pod "43bf10dc-7107-4803-a386-c98e0a9c3bc2" (UID: "43bf10dc-7107-4803-a386-c98e0a9c3bc2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:07:12.414485 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.414450 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.415276 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.415204 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.415276 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.415236 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8bgwn\" (UniqueName: \"kubernetes.io/projected/43bf10dc-7107-4803-a386-c98e0a9c3bc2-kube-api-access-8bgwn\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.415276 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.415252 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43bf10dc-7107-4803-a386-c98e0a9c3bc2-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:07:12.650299 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.650243 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh"] Oct 30 19:07:12.656264 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.656226 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-49qdh"] Oct 30 19:07:12.756505 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.756404 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq"] Oct 30 19:07:12.759530 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.759508 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-phlcq"] Oct 30 19:07:12.867803 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.867772 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv"] Oct 30 19:07:12.871645 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.871588 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-6vrbv"] Oct 30 19:07:12.981721 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.981684 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254"] Oct 30 19:07:12.985160 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:12.985134 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-1290a6-openshift-default-7f5f577cf-dd254"] Oct 30 19:07:13.520178 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.520092 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt"] Oct 30 19:07:13.520955 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.520935 5894 cpu_manager.go:414] "containerMap: removing container" podUID="43bf10dc-7107-4803-a386-c98e0a9c3bc2" containerName="istio-proxy" Oct 30 19:07:13.521059 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.520957 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="43bf10dc-7107-4803-a386-c98e0a9c3bc2" containerName="istio-proxy" Oct 30 19:07:13.521059 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.521001 5894 cpu_manager.go:414] "containerMap: removing container" podUID="e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" containerName="istio-proxy" Oct 30 19:07:13.521059 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.521009 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" containerName="istio-proxy" Oct 30 19:07:13.521227 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.521069 5894 cpu_manager.go:414] "containerMap: removing container" podUID="a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" containerName="istio-proxy" Oct 30 19:07:13.521227 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.521078 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" containerName="istio-proxy" Oct 30 19:07:13.521227 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.521091 5894 cpu_manager.go:414] "containerMap: removing container" podUID="d1d373aa-3cfa-4f6a-9746-889e0d90cde2" containerName="istio-proxy" Oct 30 19:07:13.521227 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.521098 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d1d373aa-3cfa-4f6a-9746-889e0d90cde2" containerName="istio-proxy" Oct 30 19:07:13.521405 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.521238 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" containerName="istio-proxy" Oct 30 19:07:13.521405 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.521254 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="d1d373aa-3cfa-4f6a-9746-889e0d90cde2" containerName="istio-proxy" Oct 30 19:07:13.521405 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.521266 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="43bf10dc-7107-4803-a386-c98e0a9c3bc2" containerName="istio-proxy" Oct 30 19:07:13.521405 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.521280 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" containerName="istio-proxy" Oct 30 19:07:13.526693 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.526673 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" Oct 30 19:07:13.579980 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.579948 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w3-732a3f-serving-cert\"" Oct 30 19:07:13.584040 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.584014 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt"] Oct 30 19:07:13.633907 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.633867 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9faa41c5-f9c8-4355-a8ce-d5c44409e061-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt\" (UID: \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" Oct 30 19:07:13.634057 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.634032 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9faa41c5-f9c8-4355-a8ce-d5c44409e061-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt\" (UID: \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" Oct 30 19:07:13.634347 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.634325 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjlf5\" (UniqueName: \"kubernetes.io/projected/9faa41c5-f9c8-4355-a8ce-d5c44409e061-kube-api-access-qjlf5\") pod \"maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt\" (UID: \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" Oct 30 19:07:13.634392 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.634375 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9faa41c5-f9c8-4355-a8ce-d5c44409e061-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt\" (UID: \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" Oct 30 19:07:13.654998 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.654964 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="43bf10dc-7107-4803-a386-c98e0a9c3bc2" path="/var/lib/kubelet/pods/43bf10dc-7107-4803-a386-c98e0a9c3bc2/volumes" Oct 30 19:07:13.655555 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.655532 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3" path="/var/lib/kubelet/pods/a394fd2c-8eb1-41e9-bd92-ca85d3bccbe3/volumes" Oct 30 19:07:13.656093 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.656073 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d1d373aa-3cfa-4f6a-9746-889e0d90cde2" path="/var/lib/kubelet/pods/d1d373aa-3cfa-4f6a-9746-889e0d90cde2/volumes" Oct 30 19:07:13.656954 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.656936 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb" path="/var/lib/kubelet/pods/e2b2e6c8-764f-4f40-9973-ebd7d7b1cfdb/volumes" Oct 30 19:07:13.735931 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.735896 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9faa41c5-f9c8-4355-a8ce-d5c44409e061-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt\" (UID: \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" Oct 30 19:07:13.736134 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.735972 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9faa41c5-f9c8-4355-a8ce-d5c44409e061-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt\" (UID: \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" Oct 30 19:07:13.736223 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.736204 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qjlf5\" (UniqueName: \"kubernetes.io/projected/9faa41c5-f9c8-4355-a8ce-d5c44409e061-kube-api-access-qjlf5\") pod \"maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt\" (UID: \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" Oct 30 19:07:13.736282 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.736251 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9faa41c5-f9c8-4355-a8ce-d5c44409e061-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt\" (UID: \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" Oct 30 19:07:13.736496 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:07:13.736433 5894 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-serving-cert: secret "maas-api-e2e-shared-b-w3-732a3f-serving-cert" not found Oct 30 19:07:13.736496 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:07:13.736497 5894 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9faa41c5-f9c8-4355-a8ce-d5c44409e061-maas-api-tls podName:9faa41c5-f9c8-4355-a8ce-d5c44409e061 nodeName:}" failed. No retries permitted until 2026-10-30 19:07:14.236478008 +0000 UTC m=+1704.726605827 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/9faa41c5-f9c8-4355-a8ce-d5c44409e061-maas-api-tls") pod "maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" (UID: "9faa41c5-f9c8-4355-a8ce-d5c44409e061") : secret "maas-api-e2e-shared-b-w3-732a3f-serving-cert" not found Oct 30 19:07:13.736797 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.736776 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9faa41c5-f9c8-4355-a8ce-d5c44409e061-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt\" (UID: \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" Oct 30 19:07:13.738683 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.738657 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9faa41c5-f9c8-4355-a8ce-d5c44409e061-metrics-certs\") pod \"maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt\" (UID: \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" Oct 30 19:07:13.770637 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.770498 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:13.810599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.808984 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjlf5\" (UniqueName: \"kubernetes.io/projected/9faa41c5-f9c8-4355-a8ce-d5c44409e061-kube-api-access-qjlf5\") pod \"maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt\" (UID: \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" Oct 30 19:07:13.944081 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:13.944051 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:14.125677 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:14.125603 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:14.247213 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:14.247181 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9faa41c5-f9c8-4355-a8ce-d5c44409e061-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt\" (UID: \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" Oct 30 19:07:14.250089 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:14.249437 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9faa41c5-f9c8-4355-a8ce-d5c44409e061-maas-api-tls\") pod \"maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt\" (UID: \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" Oct 30 19:07:14.440896 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:14.440861 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" Oct 30 19:07:14.820548 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:07:14.820516 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9faa41c5_f9c8_4355_a8ce_d5c44409e061.slice/crio-c56d84d1fe931adc019032ad5830c5a5c4d4229c8b47f72a2371389b579606fe WatchSource:0}: Error finding container c56d84d1fe931adc019032ad5830c5a5c4d4229c8b47f72a2371389b579606fe: Status 404 returned error can't find the container with id c56d84d1fe931adc019032ad5830c5a5c4d4229c8b47f72a2371389b579606fe Oct 30 19:07:14.843485 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:14.843458 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt"] Oct 30 19:07:15.210958 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:15.210919 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" event={"ID":"9faa41c5-f9c8-4355-a8ce-d5c44409e061","Type":"ContainerStarted","Data":"c56d84d1fe931adc019032ad5830c5a5c4d4229c8b47f72a2371389b579606fe"} Oct 30 19:07:15.212510 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:15.212486 5894 generic.go:356] "Generic (PLEG): container finished" podID="7488e3ba-340f-4d02-806d-7b22cc544066" containerID="0a138608266e629c59a2ef70e51c641345f8e090ba71e56bbead6c1c96273aff" exitCode=0 Oct 30 19:07:15.212643 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:15.212557 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" event={"ID":"7488e3ba-340f-4d02-806d-7b22cc544066","Type":"ContainerDied","Data":"0a138608266e629c59a2ef70e51c641345f8e090ba71e56bbead6c1c96273aff"} Oct 30 19:07:15.213024 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:15.213009 5894 scope.go:122] "RemoveContainer" containerID="0a138608266e629c59a2ef70e51c641345f8e090ba71e56bbead6c1c96273aff" Oct 30 19:07:15.214625 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:15.214592 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:07:15.214743 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:15.214682 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:07:15.214743 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:15.214732 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:07:15.729171 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:15.729134 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:15.963645 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:15.937902 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:16.222447 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:16.222338 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" event={"ID":"7488e3ba-340f-4d02-806d-7b22cc544066","Type":"ContainerStarted","Data":"c2884e3791b6e7fe2decd13a91f1c77c37a95f61abe20c45be0e975b757e5003"} Oct 30 19:07:16.224074 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:16.224044 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" event={"ID":"9faa41c5-f9c8-4355-a8ce-d5c44409e061","Type":"ContainerStarted","Data":"07a3d94fa8a145ae30e4899a1fd096db05312939e005fd9f70b7e3931370d22d"} Oct 30 19:07:16.224202 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:16.224175 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" Oct 30 19:07:16.460348 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:16.460296 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" podStartSLOduration=2.601293016 podStartE2EDuration="3.460277267s" totalImagesPullingTime="858.984251ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:16.456688661 +0000 UTC m=+1706.946816483" watchObservedRunningTime="2026-10-30 19:07:16.460277267 +0000 UTC m=+1706.950405118" Oct 30 19:07:16.610047 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:16.609952 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:07:16.610221 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:16.610080 5894 patch_prober.go:29] interesting pod/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body= Oct 30 19:07:16.610221 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:16.610123 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" podUID="7488e3ba-340f-4d02-806d-7b22cc544066" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" Oct 30 19:07:17.304432 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:17.304394 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:17.602234 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:17.602153 5894 patch_prober.go:29] interesting pod/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body= Oct 30 19:07:17.602234 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:17.602214 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" podUID="7488e3ba-340f-4d02-806d-7b22cc544066" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" Oct 30 19:07:18.552897 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:18.552865 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:18.601777 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:18.601743 5894 patch_prober.go:29] interesting pod/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body= Oct 30 19:07:18.601932 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:18.601796 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" podUID="7488e3ba-340f-4d02-806d-7b22cc544066" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" Oct 30 19:07:19.148640 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:19.148602 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:19.602689 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:19.602608 5894 patch_prober.go:29] interesting pod/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body= Oct 30 19:07:19.603126 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:19.603097 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" podUID="7488e3ba-340f-4d02-806d-7b22cc544066" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" Oct 30 19:07:20.118646 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:07:20.118609 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:07:20.601801 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:20.601767 5894 patch_prober.go:29] interesting pod/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body= Oct 30 19:07:20.601978 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:20.601833 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" podUID="7488e3ba-340f-4d02-806d-7b22cc544066" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" Oct 30 19:07:21.571123 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:21.571083 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:21.602384 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:21.602352 5894 patch_prober.go:29] interesting pod/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" start-of-body= Oct 30 19:07:21.602529 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:21.602403 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" podUID="7488e3ba-340f-4d02-806d-7b22cc544066" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.94:15021/healthz/ready\": dial tcp 10.132.0.94:15021: connect: connection refused" Oct 30 19:07:22.252156 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:22.252129 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" Oct 30 19:07:22.606377 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:22.606300 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:07:22.606845 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:22.606794 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:07:22.607389 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:22.607369 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:07:24.203513 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:24.203480 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:29.261638 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:29.261595 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:29.929361 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:29.929327 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:34.063767 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:34.063683 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:34.268597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:34.262311 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:34.441821 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:34.441790 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:36.014841 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:36.014801 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:36.695330 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:36.695295 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:37.400811 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.400655 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg"] Oct 30 19:07:37.433010 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.432978 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.486858 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.482653 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-envoy\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.486858 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.482693 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istiod-ca-cert\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.486858 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.482770 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-podinfo\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.486858 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.482852 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-workload-certs\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.486858 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.482892 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-token\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.486858 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.482933 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r7788\" (UniqueName: \"kubernetes.io/projected/8a237b2f-38a2-47b6-a0a4-3d89b557b419-kube-api-access-r7788\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.486858 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.483041 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-data\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.486858 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.483106 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-workload-socket\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.486858 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.483131 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-credential-socket\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.509658 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.487905 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-4ebeba-openshift-default-dockercfg-fk7k6\"" Oct 30 19:07:37.585941 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.585860 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg"] Oct 30 19:07:37.587449 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.586962 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-podinfo\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.587747 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.587725 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-workload-certs\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.587896 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.587878 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-token\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.588075 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.588054 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r7788\" (UniqueName: \"kubernetes.io/projected/8a237b2f-38a2-47b6-a0a4-3d89b557b419-kube-api-access-r7788\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.588290 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.588274 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-data\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.588442 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.588429 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-workload-socket\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.588541 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.588528 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-credential-socket\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.588776 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.588761 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-envoy\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.588956 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.588941 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istiod-ca-cert\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.589865 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.589846 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istiod-ca-cert\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.591755 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.590534 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-workload-certs\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.591883 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.590764 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-credential-socket\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.591975 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.590970 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-workload-socket\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.592066 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.591704 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-data\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.749433 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.712180 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-token\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.749433 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.719239 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:37.749433 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.728129 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r7788\" (UniqueName: \"kubernetes.io/projected/8a237b2f-38a2-47b6-a0a4-3d89b557b419-kube-api-access-r7788\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.782603 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.782101 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-podinfo\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.833002 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.830444 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-envoy\") pod \"e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:37.919083 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:37.913545 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:38.124769 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:38.124312 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:38.564627 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:38.564596 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg"] Oct 30 19:07:38.565388 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:07:38.565358 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a237b2f_38a2_47b6_a0a4_3d89b557b419.slice/crio-4602d1292a3ce03c6607c88796a34876931c74a5b18ba382948b6023333f85b5 WatchSource:0}: Error finding container 4602d1292a3ce03c6607c88796a34876931c74a5b18ba382948b6023333f85b5: Status 404 returned error can't find the container with id 4602d1292a3ce03c6607c88796a34876931c74a5b18ba382948b6023333f85b5 Oct 30 19:07:38.568150 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:38.568120 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:07:38.568236 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:38.568182 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:07:38.568236 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:38.568212 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:07:38.970374 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:38.970339 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:39.152217 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:39.152167 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:39.335693 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:39.335607 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:39.480939 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:39.480905 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" event={"ID":"8a237b2f-38a2-47b6-a0a4-3d89b557b419","Type":"ContainerStarted","Data":"398d5698699ffc986b484bf657c390586ca5833ca8ee51e124ca58cc0ff27cc8"} Oct 30 19:07:39.481121 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:39.480961 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" event={"ID":"8a237b2f-38a2-47b6-a0a4-3d89b557b419","Type":"ContainerStarted","Data":"4602d1292a3ce03c6607c88796a34876931c74a5b18ba382948b6023333f85b5"} Oct 30 19:07:39.650389 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:39.650339 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" podStartSLOduration=2.650323595 podStartE2EDuration="2.650323595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:39.647995883 +0000 UTC m=+1730.138123703" watchObservedRunningTime="2026-10-30 19:07:39.650323595 +0000 UTC m=+1730.140451419" Oct 30 19:07:40.125023 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:40.124938 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:40.130149 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:40.130124 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:40.489812 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:40.489779 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:40.490693 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:40.490673 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:07:43.070838 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:43.070809 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:43.265619 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:43.265556 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:44.109601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:44.105638 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:44.313937 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:44.313880 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:44.505713 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:44.505686 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:46.490321 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:46.490281 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:46.671594 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:46.671536 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:50.118633 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:07:50.118592 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:07:50.211608 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:50.209655 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:52.352317 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:52.352231 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:52.622651 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:52.611824 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:52.733913 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:52.732751 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4"] Oct 30 19:07:52.756355 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:52.756306 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4" Oct 30 19:07:52.804601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:52.803885 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4"] Oct 30 19:07:52.836221 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:52.836195 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-4ebeba\"" Oct 30 19:07:52.836818 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:52.836791 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-4ebeba-dockercfg-btvpm\"" Oct 30 19:07:52.973176 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:52.973134 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h8z5b\" (UniqueName: \"kubernetes.io/projected/e56af8be-26ce-45af-963b-43797d0c0881-kube-api-access-h8z5b\") pod \"payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4\" (UID: \"e56af8be-26ce-45af-963b-43797d0c0881\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4" Oct 30 19:07:52.980238 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:52.973676 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e56af8be-26ce-45af-963b-43797d0c0881-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4\" (UID: \"e56af8be-26ce-45af-963b-43797d0c0881\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4" Oct 30 19:07:53.097598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:53.090736 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h8z5b\" (UniqueName: \"kubernetes.io/projected/e56af8be-26ce-45af-963b-43797d0c0881-kube-api-access-h8z5b\") pod \"payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4\" (UID: \"e56af8be-26ce-45af-963b-43797d0c0881\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4" Oct 30 19:07:53.126612 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:53.098056 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e56af8be-26ce-45af-963b-43797d0c0881-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4\" (UID: \"e56af8be-26ce-45af-963b-43797d0c0881\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4" Oct 30 19:07:53.190601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:53.171037 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e56af8be-26ce-45af-963b-43797d0c0881-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4\" (UID: \"e56af8be-26ce-45af-963b-43797d0c0881\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4" Oct 30 19:07:53.250173 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:53.214327 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h8z5b\" (UniqueName: \"kubernetes.io/projected/e56af8be-26ce-45af-963b-43797d0c0881-kube-api-access-h8z5b\") pod \"payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4\" (UID: \"e56af8be-26ce-45af-963b-43797d0c0881\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4" Oct 30 19:07:53.480304 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:53.480275 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4" Oct 30 19:07:53.856596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:53.856556 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4"] Oct 30 19:07:53.858565 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:07:53.858536 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode56af8be_26ce_45af_963b_43797d0c0881.slice/crio-16325e3a63a4e155cac32ef5362b3c30f38cee4fdf1e3a373a07c07abf3aa249 WatchSource:0}: Error finding container 16325e3a63a4e155cac32ef5362b3c30f38cee4fdf1e3a373a07c07abf3aa249: Status 404 returned error can't find the container with id 16325e3a63a4e155cac32ef5362b3c30f38cee4fdf1e3a373a07c07abf3aa249 Oct 30 19:07:54.742213 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:54.742174 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4" event={"ID":"e56af8be-26ce-45af-963b-43797d0c0881","Type":"ContainerStarted","Data":"05bd3f4f2f2f9bdad61809b0ed6cd26b39e13ad21188a7ee7c02a0226e200b97"} Oct 30 19:07:54.742683 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:54.742236 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4" Oct 30 19:07:54.742683 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:54.742256 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4" event={"ID":"e56af8be-26ce-45af-963b-43797d0c0881","Type":"ContainerStarted","Data":"16325e3a63a4e155cac32ef5362b3c30f38cee4fdf1e3a373a07c07abf3aa249"} Oct 30 19:07:54.856647 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:54.856599 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4" podStartSLOduration=2.856585425 podStartE2EDuration="2.856585425s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:07:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:07:54.854824726 +0000 UTC m=+1745.344952557" watchObservedRunningTime="2026-10-30 19:07:54.856585425 +0000 UTC m=+1745.346713237" Oct 30 19:07:55.029601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:55.029451 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:07:59.726909 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:07:59.726874 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:02.119592 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.119546 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw"] Oct 30 19:08:02.124025 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.124007 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.171118 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.171084 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv"] Oct 30 19:08:02.175658 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.175639 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb"] Oct 30 19:08:02.175785 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.175763 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.180128 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.180103 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.222096 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.222062 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw"] Oct 30 19:08:02.280628 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.280594 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb"] Oct 30 19:08:02.291073 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.291040 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.291264 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.291099 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.291264 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.291201 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.291264 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.291245 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.291422 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.291298 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.291422 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.291335 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.291422 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.291353 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a3b4479-829a-4ac7-8756-52988b12386d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.291596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.291437 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf743c6e-7543-408a-b557-cdf069860419-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.291596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.291496 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.291596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.291531 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.291750 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.291620 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p8hh\" (UniqueName: \"kubernetes.io/projected/81eabfd4-ebb5-43da-9d2c-03137dada16f-kube-api-access-6p8hh\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.291750 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.291666 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.291750 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.291691 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.291896 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.291740 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf743c6e-7543-408a-b557-cdf069860419-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.291896 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.291793 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.291896 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.291834 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.291896 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.291888 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjnw8\" (UniqueName: \"kubernetes.io/projected/cf743c6e-7543-408a-b557-cdf069860419-kube-api-access-vjnw8\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.292095 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.291956 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.292095 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.291986 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.292095 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.292006 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a3b4479-829a-4ac7-8756-52988b12386d-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.292095 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.292042 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81eabfd4-ebb5-43da-9d2c-03137dada16f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.292289 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.292147 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a3b4479-829a-4ac7-8756-52988b12386d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.292289 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.292186 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvhfn\" (UniqueName: \"kubernetes.io/projected/7a3b4479-829a-4ac7-8756-52988b12386d-kube-api-access-tvhfn\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.292289 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.292225 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf743c6e-7543-408a-b557-cdf069860419-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.292289 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.292267 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.292545 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.292312 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.292545 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.292340 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.326896 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.326863 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv"] Oct 30 19:08:02.393861 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.393825 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.394054 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.393874 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a3b4479-829a-4ac7-8756-52988b12386d-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.394054 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.393933 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81eabfd4-ebb5-43da-9d2c-03137dada16f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.394054 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.393975 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a3b4479-829a-4ac7-8756-52988b12386d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.394054 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394010 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvhfn\" (UniqueName: \"kubernetes.io/projected/7a3b4479-829a-4ac7-8756-52988b12386d-kube-api-access-tvhfn\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.394054 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394037 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf743c6e-7543-408a-b557-cdf069860419-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.394343 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394079 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.394343 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394116 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.394343 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394204 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.394343 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394247 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.394343 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394312 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.394675 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394339 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.394675 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394354 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.394675 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394419 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.394675 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394473 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.394675 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394527 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.394675 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394554 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a3b4479-829a-4ac7-8756-52988b12386d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.394675 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394644 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf743c6e-7543-408a-b557-cdf069860419-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.394675 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394671 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.395081 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394699 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.395081 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394743 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p8hh\" (UniqueName: \"kubernetes.io/projected/81eabfd4-ebb5-43da-9d2c-03137dada16f-kube-api-access-6p8hh\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.395081 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394811 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a3b4479-829a-4ac7-8756-52988b12386d-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.395081 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394826 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.395081 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394886 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.395081 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394922 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf743c6e-7543-408a-b557-cdf069860419-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.395081 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394954 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.395081 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.394993 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.395081 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.395069 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjnw8\" (UniqueName: \"kubernetes.io/projected/cf743c6e-7543-408a-b557-cdf069860419-kube-api-access-vjnw8\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.395591 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.395127 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.395591 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.395164 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.395591 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.395198 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.395591 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.395488 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.395591 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.395521 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.395864 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.395830 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.397089 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.396167 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf743c6e-7543-408a-b557-cdf069860419-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.397089 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.396266 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.397089 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.396612 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.397089 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.396817 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.397089 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.396858 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.397089 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.396940 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.397089 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.397049 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.398695 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.398513 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.398695 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.398650 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81eabfd4-ebb5-43da-9d2c-03137dada16f-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.400648 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.400623 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.400765 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.400748 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.401056 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.401035 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf743c6e-7543-408a-b557-cdf069860419-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.401652 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.401557 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.402793 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.402772 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a3b4479-829a-4ac7-8756-52988b12386d-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.453292 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.453184 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf743c6e-7543-408a-b557-cdf069860419-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.453292 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.453266 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p8hh\" (UniqueName: \"kubernetes.io/projected/81eabfd4-ebb5-43da-9d2c-03137dada16f-kube-api-access-6p8hh\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.453778 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.453754 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-jk6fw\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.454261 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.454241 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvhfn\" (UniqueName: \"kubernetes.io/projected/7a3b4479-829a-4ac7-8756-52988b12386d-kube-api-access-tvhfn\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.454681 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.454659 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjnw8\" (UniqueName: \"kubernetes.io/projected/cf743c6e-7543-408a-b557-cdf069860419-kube-api-access-vjnw8\") pod \"maas-default-gateway-openshift-default-6db586dd6c-2vwtv\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.454913 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.454891 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a3b4479-829a-4ac7-8756-52988b12386d-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-rsntb\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.492939 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.492908 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:02.499383 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.499361 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:02.739432 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.739344 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:02.925179 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.925146 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv"] Oct 30 19:08:02.925367 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:08:02.925345 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcf743c6e_7543_408a_b557_cdf069860419.slice/crio-99bf07fc6ecf5c267134593004506dcf520d5bdf0c60d4e506ba6c6e48db7721 WatchSource:0}: Error finding container 99bf07fc6ecf5c267134593004506dcf520d5bdf0c60d4e506ba6c6e48db7721: Status 404 returned error can't find the container with id 99bf07fc6ecf5c267134593004506dcf520d5bdf0c60d4e506ba6c6e48db7721 Oct 30 19:08:02.927959 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.927929 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:08:02.928055 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.927988 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:08:02.928055 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.928018 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:08:02.928503 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.928438 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb"] Oct 30 19:08:02.966092 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:08:02.966066 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a3b4479_829a_4ac7_8756_52988b12386d.slice/crio-eb3a0b9d118bdfd3ffc41eb5a02aba874feb0935239db4cc3f88af2e6b6c30b1 WatchSource:0}: Error finding container eb3a0b9d118bdfd3ffc41eb5a02aba874feb0935239db4cc3f88af2e6b6c30b1: Status 404 returned error can't find the container with id eb3a0b9d118bdfd3ffc41eb5a02aba874feb0935239db4cc3f88af2e6b6c30b1 Oct 30 19:08:02.969546 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.969500 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:08:02.969679 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.969565 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:08:02.969679 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:02.969635 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:08:03.121953 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:03.121921 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw"] Oct 30 19:08:03.122471 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:08:03.121987 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81eabfd4_ebb5_43da_9d2c_03137dada16f.slice/crio-d1bafd976f340ed65d908c800fbd9711200f4f351af027edfdceeb5e1fb109ef WatchSource:0}: Error finding container d1bafd976f340ed65d908c800fbd9711200f4f351af027edfdceeb5e1fb109ef: Status 404 returned error can't find the container with id d1bafd976f340ed65d908c800fbd9711200f4f351af027edfdceeb5e1fb109ef Oct 30 19:08:03.126333 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:03.126292 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:08:03.126458 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:03.126380 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:08:03.126458 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:03.126421 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:08:03.837489 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:03.837449 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" event={"ID":"81eabfd4-ebb5-43da-9d2c-03137dada16f","Type":"ContainerStarted","Data":"33334ea7b59d34dc126dced8a4a04187a8cc4c934a031e477e36a502d33963ed"} Oct 30 19:08:03.837779 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:03.837547 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" event={"ID":"81eabfd4-ebb5-43da-9d2c-03137dada16f","Type":"ContainerStarted","Data":"d1bafd976f340ed65d908c800fbd9711200f4f351af027edfdceeb5e1fb109ef"} Oct 30 19:08:03.839370 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:03.839335 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" event={"ID":"cf743c6e-7543-408a-b557-cdf069860419","Type":"ContainerStarted","Data":"5a761ebb66c6a2af78c59dd8a4740811c98092e7495c2579ce52e1e063c830d4"} Oct 30 19:08:03.839495 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:03.839393 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" event={"ID":"cf743c6e-7543-408a-b557-cdf069860419","Type":"ContainerStarted","Data":"99bf07fc6ecf5c267134593004506dcf520d5bdf0c60d4e506ba6c6e48db7721"} Oct 30 19:08:03.842245 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:03.842207 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" event={"ID":"7a3b4479-829a-4ac7-8756-52988b12386d","Type":"ContainerStarted","Data":"5e405111e93065ee9022955a062ce09d0447d3d6b212f9ec067d418c04c55f1d"} Oct 30 19:08:03.842245 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:03.842264 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" event={"ID":"7a3b4479-829a-4ac7-8756-52988b12386d","Type":"ContainerStarted","Data":"eb3a0b9d118bdfd3ffc41eb5a02aba874feb0935239db4cc3f88af2e6b6c30b1"} Oct 30 19:08:03.965613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:03.962206 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" podStartSLOduration=1.962184743 podStartE2EDuration="1.962184743s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:03.956221683 +0000 UTC m=+1754.446349546" watchObservedRunningTime="2026-10-30 19:08:03.962184743 +0000 UTC m=+1754.452312564" Oct 30 19:08:04.092658 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:04.089465 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" podStartSLOduration=2.089442247 podStartE2EDuration="2.089442247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:04.074939949 +0000 UTC m=+1754.565067771" watchObservedRunningTime="2026-10-30 19:08:04.089442247 +0000 UTC m=+1754.579570067" Oct 30 19:08:04.203565 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:04.203483 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" podStartSLOduration=2.203463234 podStartE2EDuration="2.203463234s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:08:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:08:04.200115829 +0000 UTC m=+1754.690243652" watchObservedRunningTime="2026-10-30 19:08:04.203463234 +0000 UTC m=+1754.693591055" Oct 30 19:08:04.502600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:04.501973 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:04.502600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:04.502320 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:04.506598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:04.505672 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-2vwtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body= Oct 30 19:08:04.506598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:04.505726 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" podUID="cf743c6e-7543-408a-b557-cdf069860419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" Oct 30 19:08:04.506598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:04.506026 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-rsntb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body= Oct 30 19:08:04.506598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:04.506052 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" podUID="7a3b4479-829a-4ac7-8756-52988b12386d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" Oct 30 19:08:04.550595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:04.538623 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:04.744599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:04.743779 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:04.744599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:04.743907 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-jk6fw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body= Oct 30 19:08:04.744599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:04.743948 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" podUID="81eabfd4-ebb5-43da-9d2c-03137dada16f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" Oct 30 19:08:05.505595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:05.501975 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-2vwtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body= Oct 30 19:08:05.505595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:05.502055 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" podUID="cf743c6e-7543-408a-b557-cdf069860419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" Oct 30 19:08:05.505595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:05.503241 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-rsntb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body= Oct 30 19:08:05.505595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:05.503284 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" podUID="7a3b4479-829a-4ac7-8756-52988b12386d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" Oct 30 19:08:05.751724 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:05.751677 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-jk6fw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body= Oct 30 19:08:05.752013 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:05.751980 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" podUID="81eabfd4-ebb5-43da-9d2c-03137dada16f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" Oct 30 19:08:05.802652 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:05.801744 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4" Oct 30 19:08:06.502599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:06.501995 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-rsntb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body= Oct 30 19:08:06.502599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:06.502076 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" podUID="7a3b4479-829a-4ac7-8756-52988b12386d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" Oct 30 19:08:06.504614 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:06.503694 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-2vwtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body= Oct 30 19:08:06.504614 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:06.503747 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" podUID="cf743c6e-7543-408a-b557-cdf069860419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" Oct 30 19:08:06.752423 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:06.752387 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-jk6fw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body= Oct 30 19:08:06.753180 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:06.753100 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" podUID="81eabfd4-ebb5-43da-9d2c-03137dada16f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" Oct 30 19:08:07.496599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:07.495828 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-2vwtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body= Oct 30 19:08:07.496599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:07.495894 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" podUID="cf743c6e-7543-408a-b557-cdf069860419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" Oct 30 19:08:07.504638 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:07.499892 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-rsntb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body= Oct 30 19:08:07.504638 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:07.499942 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" podUID="7a3b4479-829a-4ac7-8756-52988b12386d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" Oct 30 19:08:07.741556 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:07.741516 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-jk6fw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body= Oct 30 19:08:07.741838 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:07.741807 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" podUID="81eabfd4-ebb5-43da-9d2c-03137dada16f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" Oct 30 19:08:08.496597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:08.494107 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-2vwtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body= Oct 30 19:08:08.496597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:08.494196 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" podUID="cf743c6e-7543-408a-b557-cdf069860419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" Oct 30 19:08:08.514599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:08.502520 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-rsntb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body= Oct 30 19:08:08.514599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:08.502641 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" podUID="7a3b4479-829a-4ac7-8756-52988b12386d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" Oct 30 19:08:08.647601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:08.632685 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" podUID="ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:08:08.690599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:08.689971 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:08:08.690599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:08.690389 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:08:08.695598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:08.691556 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:08:08.696928 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:08.696893 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:08:08.697409 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:08.697381 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded" Oct 30 19:08:08.745007 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:08.744967 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-jk6fw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body= Oct 30 19:08:08.746555 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:08.746503 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" podUID="81eabfd4-ebb5-43da-9d2c-03137dada16f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" Oct 30 19:08:09.496601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:09.493921 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-2vwtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body= Oct 30 19:08:09.496601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:09.493995 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" podUID="cf743c6e-7543-408a-b557-cdf069860419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" Oct 30 19:08:09.502808 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:09.502312 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-rsntb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body= Oct 30 19:08:09.502808 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:09.502371 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" podUID="7a3b4479-829a-4ac7-8756-52988b12386d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" Oct 30 19:08:09.756231 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:09.756139 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-jk6fw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body= Oct 30 19:08:09.756562 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:09.756531 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" podUID="81eabfd4-ebb5-43da-9d2c-03137dada16f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" Oct 30 19:08:10.499755 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:10.494022 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-2vwtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body= Oct 30 19:08:10.499755 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:10.494083 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" podUID="cf743c6e-7543-408a-b557-cdf069860419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" Oct 30 19:08:10.515609 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:10.511032 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-rsntb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body= Oct 30 19:08:10.515609 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:10.511094 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" podUID="7a3b4479-829a-4ac7-8756-52988b12386d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" Oct 30 19:08:10.576644 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:10.571253 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:10.862681 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:10.860004 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-jk6fw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body= Oct 30 19:08:10.862681 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:10.860063 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" podUID="81eabfd4-ebb5-43da-9d2c-03137dada16f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" Oct 30 19:08:11.493845 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:11.493801 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-2vwtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body= Oct 30 19:08:11.494058 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:11.493870 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" podUID="cf743c6e-7543-408a-b557-cdf069860419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" Oct 30 19:08:11.500154 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:11.500121 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-rsntb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body= Oct 30 19:08:11.500625 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:11.500169 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" podUID="7a3b4479-829a-4ac7-8756-52988b12386d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" Oct 30 19:08:11.740707 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:11.740664 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-jk6fw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body= Oct 30 19:08:11.740906 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:11.740729 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" podUID="81eabfd4-ebb5-43da-9d2c-03137dada16f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" Oct 30 19:08:12.502597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:12.501875 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-rsntb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" start-of-body= Oct 30 19:08:12.502597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:12.502037 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" podUID="7a3b4479-829a-4ac7-8756-52988b12386d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.103:15021/healthz/ready\": dial tcp 10.132.0.103:15021: connect: connection refused" Oct 30 19:08:12.503135 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:12.503081 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-2vwtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" start-of-body= Oct 30 19:08:12.503196 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:12.503134 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" podUID="cf743c6e-7543-408a-b557-cdf069860419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": dial tcp 10.132.0.104:15021: connect: connection refused" Oct 30 19:08:12.510326 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:12.509070 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:12.693012 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:12.692970 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:12.740372 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:12.740340 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-jk6fw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" start-of-body= Oct 30 19:08:12.740641 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:12.740609 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" podUID="81eabfd4-ebb5-43da-9d2c-03137dada16f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.102:15021/healthz/ready\": dial tcp 10.132.0.102:15021: connect: connection refused" Oct 30 19:08:14.243940 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:14.242458 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:14.243940 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:14.243799 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:14.245069 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:14.245022 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:08:14.494027 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:14.493917 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-2vwtv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.104:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:08:14.494027 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:14.493986 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" podUID="cf743c6e-7543-408a-b557-cdf069860419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.104:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:08:14.516619 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:14.516556 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:14.517987 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:14.517853 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:14.518827 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:14.518800 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:08:14.636934 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:14.636657 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:14.637608 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:14.637543 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:14.638552 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:14.638531 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:08:15.469647 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:15.469618 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:20.119273 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:08:20.119235 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:08:20.312123 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:20.312083 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:21.932223 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:21.932181 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:25.037949 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:25.037853 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:29.768597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:29.764878 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:33.928640 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:33.903042 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:35.255544 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:35.255505 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:35.696587 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:35.696550 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:36.088976 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:36.088886 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:36.297196 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:36.297162 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:36.494185 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:36.494151 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:37.381470 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:37.381426 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:37.778599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:37.758839 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:38.026513 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:38.026471 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7"] Oct 30 19:08:38.027053 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:38.027017 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" podUID="7488e3ba-340f-4d02-806d-7b22cc544066" containerName="istio-proxy" containerID="cri-o://c2884e3791b6e7fe2decd13a91f1c77c37a95f61abe20c45be0e975b757e5003" gracePeriod=30 Oct 30 19:08:38.057008 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:38.056878 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:41.456016 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:41.455979 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:41.667153 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:41.667116 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:41.942380 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:41.942327 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:42.730229 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:42.730199 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:43.489717 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:43.467895 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:43.645606 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:43.645054 5894 generic.go:356] "Generic (PLEG): container finished" podID="7488e3ba-340f-4d02-806d-7b22cc544066" containerID="c2884e3791b6e7fe2decd13a91f1c77c37a95f61abe20c45be0e975b757e5003" exitCode=0 Oct 30 19:08:43.659398 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:43.659345 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" event={"ID":"7488e3ba-340f-4d02-806d-7b22cc544066","Type":"ContainerDied","Data":"c2884e3791b6e7fe2decd13a91f1c77c37a95f61abe20c45be0e975b757e5003"} Oct 30 19:08:43.659906 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:43.659856 5894 scope.go:122] "RemoveContainer" containerID="0a138608266e629c59a2ef70e51c641345f8e090ba71e56bbead6c1c96273aff" Oct 30 19:08:43.951750 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:43.951717 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:08:44.009600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.005499 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:44.050266 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.050229 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-istio-data\") pod \"7488e3ba-340f-4d02-806d-7b22cc544066\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " Oct 30 19:08:44.050788 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.050758 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-workload-certs\") pod \"7488e3ba-340f-4d02-806d-7b22cc544066\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " Oct 30 19:08:44.051067 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.051044 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7488e3ba-340f-4d02-806d-7b22cc544066-istio-token\" (UniqueName: \"kubernetes.io/projected/7488e3ba-340f-4d02-806d-7b22cc544066-istio-token\") pod \"7488e3ba-340f-4d02-806d-7b22cc544066\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " Oct 30 19:08:44.051375 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.051350 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-credential-socket\") pod \"7488e3ba-340f-4d02-806d-7b22cc544066\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " Oct 30 19:08:44.051561 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.051545 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-workload-socket\") pod \"7488e3ba-340f-4d02-806d-7b22cc544066\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " Oct 30 19:08:44.070621 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.052196 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-workload-socket" pod "7488e3ba-340f-4d02-806d-7b22cc544066" (UID: "7488e3ba-340f-4d02-806d-7b22cc544066"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:08:44.070621 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.052803 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-istio-data" pod "7488e3ba-340f-4d02-806d-7b22cc544066" (UID: "7488e3ba-340f-4d02-806d-7b22cc544066"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:08:44.070621 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.052989 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-credential-socket" pod "7488e3ba-340f-4d02-806d-7b22cc544066" (UID: "7488e3ba-340f-4d02-806d-7b22cc544066"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:08:44.071289 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.071213 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7488e3ba-340f-4d02-806d-7b22cc544066-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7488e3ba-340f-4d02-806d-7b22cc544066-istiod-ca-cert\") pod \"7488e3ba-340f-4d02-806d-7b22cc544066\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " Oct 30 19:08:44.071833 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.071636 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-workload-certs" pod "7488e3ba-340f-4d02-806d-7b22cc544066" (UID: "7488e3ba-340f-4d02-806d-7b22cc544066"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:08:44.071833 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.071687 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7488e3ba-340f-4d02-806d-7b22cc544066-kube-api-access-br4z9\" (UniqueName: \"kubernetes.io/projected/7488e3ba-340f-4d02-806d-7b22cc544066-kube-api-access-br4z9\") pod \"7488e3ba-340f-4d02-806d-7b22cc544066\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " Oct 30 19:08:44.072502 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.072476 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-istio-envoy\") pod \"7488e3ba-340f-4d02-806d-7b22cc544066\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " Oct 30 19:08:44.072881 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.072754 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7488e3ba-340f-4d02-806d-7b22cc544066-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7488e3ba-340f-4d02-806d-7b22cc544066-istio-podinfo\") pod \"7488e3ba-340f-4d02-806d-7b22cc544066\" (UID: \"7488e3ba-340f-4d02-806d-7b22cc544066\") " Oct 30 19:08:44.074148 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.074129 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:08:44.074294 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.074279 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:08:44.074401 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.074389 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:08:44.074510 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.074499 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:08:44.079206 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.079124 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-istio-envoy" pod "7488e3ba-340f-4d02-806d-7b22cc544066" (UID: "7488e3ba-340f-4d02-806d-7b22cc544066"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:08:44.096520 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.093548 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7488e3ba-340f-4d02-806d-7b22cc544066-istio-token" pod "7488e3ba-340f-4d02-806d-7b22cc544066" (UID: "7488e3ba-340f-4d02-806d-7b22cc544066"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:08:44.096520 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.093668 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7488e3ba-340f-4d02-806d-7b22cc544066-kube-api-access-br4z9" pod "7488e3ba-340f-4d02-806d-7b22cc544066" (UID: "7488e3ba-340f-4d02-806d-7b22cc544066"). InnerVolumeSpecName "kube-api-access-br4z9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:08:44.100591 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.098912 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7488e3ba-340f-4d02-806d-7b22cc544066-istio-podinfo" pod "7488e3ba-340f-4d02-806d-7b22cc544066" (UID: "7488e3ba-340f-4d02-806d-7b22cc544066"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:08:44.103026 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.102985 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7488e3ba-340f-4d02-806d-7b22cc544066-istiod-ca-cert" pod "7488e3ba-340f-4d02-806d-7b22cc544066" (UID: "7488e3ba-340f-4d02-806d-7b22cc544066"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:08:44.176138 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.176099 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7488e3ba-340f-4d02-806d-7b22cc544066-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:08:44.176138 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.176140 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7488e3ba-340f-4d02-806d-7b22cc544066-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:08:44.176384 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.176156 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-br4z9\" (UniqueName: \"kubernetes.io/projected/7488e3ba-340f-4d02-806d-7b22cc544066-kube-api-access-br4z9\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:08:44.176384 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.176171 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7488e3ba-340f-4d02-806d-7b22cc544066-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:08:44.176384 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.176185 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7488e3ba-340f-4d02-806d-7b22cc544066-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:08:44.622968 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.622935 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:44.668548 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.668499 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" event={"ID":"7488e3ba-340f-4d02-806d-7b22cc544066","Type":"ContainerDied","Data":"dc728dd52a361a98b3289ae2354d2f9d02535928fa667425a4b888592e0ba179"} Oct 30 19:08:44.668758 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.668639 5894 scope.go:122] "RemoveContainer" containerID="c2884e3791b6e7fe2decd13a91f1c77c37a95f61abe20c45be0e975b757e5003" Oct 30 19:08:44.668844 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.668772 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7" Oct 30 19:08:44.818907 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.818869 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:44.844158 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.844094 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7"] Oct 30 19:08:44.850221 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:44.850172 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-eb4c249f-openshift-default-54dbb6667-84sf7"] Oct 30 19:08:45.652947 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:45.652915 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7488e3ba-340f-4d02-806d-7b22cc544066" path="/var/lib/kubelet/pods/7488e3ba-340f-4d02-806d-7b22cc544066/volumes" Oct 30 19:08:45.869124 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:45.868689 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": read tcp 10.132.0.2:46322->10.132.0.96:15021: read: connection reset by peer" start-of-body= Oct 30 19:08:45.869124 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:45.868825 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" podUID="8db11496-7f7c-49f1-ba40-08248a5b2db2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": read tcp 10.132.0.2:46322->10.132.0.96:15021: read: connection reset by peer" Oct 30 19:08:46.688433 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:46.688400 5894 generic.go:356] "Generic (PLEG): container finished" podID="8db11496-7f7c-49f1-ba40-08248a5b2db2" containerID="55a641a7e5e99592f04a6527fa9239ffdc56443ae313778feaa06dd679f9ef81" exitCode=0 Oct 30 19:08:46.688856 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:46.688481 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" event={"ID":"8db11496-7f7c-49f1-ba40-08248a5b2db2","Type":"ContainerDied","Data":"55a641a7e5e99592f04a6527fa9239ffdc56443ae313778feaa06dd679f9ef81"} Oct 30 19:08:46.688932 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:46.688861 5894 scope.go:122] "RemoveContainer" containerID="55a641a7e5e99592f04a6527fa9239ffdc56443ae313778feaa06dd679f9ef81" Oct 30 19:08:46.690543 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:46.690517 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:08:46.690677 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:46.690605 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:08:46.690677 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:46.690643 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:08:47.548598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:47.545115 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:47.698499 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:47.698460 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" event={"ID":"8db11496-7f7c-49f1-ba40-08248a5b2db2","Type":"ContainerStarted","Data":"06aa1bf9f0975c86014627242a56ca2116df6e9bbb07afbca7bcd409907588d2"} Oct 30 19:08:47.757173 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:47.757104 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:47.972751 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:47.972707 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:48.164099 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:48.164065 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:08:48.164284 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:48.164256 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body= Oct 30 19:08:48.164340 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:48.164313 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" podUID="8db11496-7f7c-49f1-ba40-08248a5b2db2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" Oct 30 19:08:49.163781 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:49.163745 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body= Oct 30 19:08:49.164182 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:49.163803 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" podUID="8db11496-7f7c-49f1-ba40-08248a5b2db2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" Oct 30 19:08:49.887756 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:49.887728 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 19:08:49.887963 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:49.887734 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 19:08:50.119105 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:08:50.119069 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:08:50.164199 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:50.164166 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body= Oct 30 19:08:50.164620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:50.164217 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" podUID="8db11496-7f7c-49f1-ba40-08248a5b2db2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" Oct 30 19:08:50.955201 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:50.955163 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:51.180596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:51.156164 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:51.222721 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:51.199741 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body= Oct 30 19:08:51.269744 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:51.269691 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" podUID="8db11496-7f7c-49f1-ba40-08248a5b2db2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" Oct 30 19:08:52.185603 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:52.179710 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body= Oct 30 19:08:52.185603 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:52.179795 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" podUID="8db11496-7f7c-49f1-ba40-08248a5b2db2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" Oct 30 19:08:53.173600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:53.173045 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body= Oct 30 19:08:53.173600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:53.173124 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" podUID="8db11496-7f7c-49f1-ba40-08248a5b2db2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" Oct 30 19:08:53.253627 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:53.247111 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:53.841614 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:53.841553 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:54.166027 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:54.165990 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" start-of-body= Oct 30 19:08:54.166492 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:54.166458 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" podUID="8db11496-7f7c-49f1-ba40-08248a5b2db2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.96:15021/healthz/ready\": dial tcp 10.132.0.96:15021: connect: connection refused" Oct 30 19:08:54.753171 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:54.753133 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:54.966787 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:54.966754 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:55.168404 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:55.168373 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:08:55.168704 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:55.168682 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:08:55.169380 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:55.169361 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:08:55.360696 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:55.360664 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:58.820899 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:58.820822 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:59.116214 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:59.116121 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:08:59.330551 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:08:59.330515 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:02.173658 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:02.166322 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:02.357237 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:02.357175 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:03.013299 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:03.013195 5894 generic.go:356] "Generic (PLEG): container finished" podID="ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" containerID="2ef858e03d4bb08b4e636b00ce8def883e42d62a73c340fbcd1ecd0236b88406" exitCode=0 Oct 30 19:09:03.013299 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:03.013262 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" event={"ID":"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4","Type":"ContainerDied","Data":"2ef858e03d4bb08b4e636b00ce8def883e42d62a73c340fbcd1ecd0236b88406"} Oct 30 19:09:03.013750 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:03.013728 5894 scope.go:122] "RemoveContainer" containerID="2ef858e03d4bb08b4e636b00ce8def883e42d62a73c340fbcd1ecd0236b88406" Oct 30 19:09:03.015880 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:03.015851 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:09:03.015995 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:03.015922 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:09:03.015995 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:03.015954 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:09:03.148214 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:03.148157 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:04.021894 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:04.021860 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" event={"ID":"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4","Type":"ContainerStarted","Data":"404b498bc23c0cc8327529f63423b3caaaab0affc8224cc092465328d8890b1d"} Oct 30 19:09:04.088229 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:04.088186 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:09:04.088429 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:04.088315 5894 patch_prober.go:29] interesting pod/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body= Oct 30 19:09:04.088429 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:04.088373 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" podUID="ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" Oct 30 19:09:04.199507 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:04.199448 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:05.085758 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:05.085727 5894 patch_prober.go:29] interesting pod/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body= Oct 30 19:09:05.086140 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:05.085772 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" podUID="ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" Oct 30 19:09:06.085912 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:06.085867 5894 patch_prober.go:29] interesting pod/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body= Oct 30 19:09:06.086316 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:06.085936 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" podUID="ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" Oct 30 19:09:06.869930 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:06.869893 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s"] Oct 30 19:09:06.871821 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:06.871791 5894 cpu_manager.go:414] "containerMap: removing container" podUID="7488e3ba-340f-4d02-806d-7b22cc544066" containerName="istio-proxy" Oct 30 19:09:06.871821 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:06.871822 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7488e3ba-340f-4d02-806d-7b22cc544066" containerName="istio-proxy" Oct 30 19:09:06.872077 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:06.872043 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="7488e3ba-340f-4d02-806d-7b22cc544066" containerName="istio-proxy" Oct 30 19:09:06.875275 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:06.875252 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:06.905608 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:06.905559 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48092d6e-157f-4df3-828b-497746965c69-tls-certs\") pod \"e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:06.905762 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:06.905627 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-home\") pod \"e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:06.905762 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:06.905742 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-dshm\") pod \"e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:06.905890 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:06.905847 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-model-cache\") pod \"e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:06.905950 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:06.905887 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-tmp-dir\") pod \"e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:06.906039 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:06.906023 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvwrk\" (UniqueName: \"kubernetes.io/projected/48092d6e-157f-4df3-828b-497746965c69-kube-api-access-cvwrk\") pod \"e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:06.925942 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:06.925912 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s"] Oct 30 19:09:06.961772 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:06.961723 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:07.005331 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:07.005282 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ae6a138d\"/\"openshift-service-ca.crt\"" Oct 30 19:09:07.005501 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:07.005440 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-ae6a138d\"/\"kube-root-ca.crt\"" Oct 30 19:09:07.005649 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:07.005547 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ae6a138d\"/\"e2e-lifecycle-model-ae6a138d-kserve-self-signed-certs\"" Oct 30 19:09:07.005722 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:07.005650 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-ae6a138d\"/\"default-dockercfg-g9kzx\"" Oct 30 19:09:07.136652 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:07.135609 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-model-cache\") pod \"e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:07.136652 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:07.135678 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-tmp-dir\") pod \"e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:07.136652 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:07.135813 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvwrk\" (UniqueName: \"kubernetes.io/projected/48092d6e-157f-4df3-828b-497746965c69-kube-api-access-cvwrk\") pod \"e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:07.136652 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:07.135843 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48092d6e-157f-4df3-828b-497746965c69-tls-certs\") pod \"e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:07.136652 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:07.135880 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-home\") pod \"e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:07.169599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:07.141673 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-dshm\") pod \"e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:07.189653 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:07.174968 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-home\") pod \"e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:07.189653 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:07.175375 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-model-cache\") pod \"e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:07.189653 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:07.178231 5894 patch_prober.go:29] interesting pod/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body= Oct 30 19:09:07.189653 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:07.178286 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" podUID="ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" Oct 30 19:09:07.189653 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:07.178938 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-tmp-dir\") pod \"e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:07.191608 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:07.191127 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-dshm\") pod \"e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:07.193523 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:07.193081 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48092d6e-157f-4df3-828b-497746965c69-tls-certs\") pod \"e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:07.287598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:07.278859 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvwrk\" (UniqueName: \"kubernetes.io/projected/48092d6e-157f-4df3-828b-497746965c69-kube-api-access-cvwrk\") pod \"e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:07.532598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:07.515956 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:08.083967 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:08.083940 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s"] Oct 30 19:09:08.085168 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:08.085129 5894 patch_prober.go:29] interesting pod/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body= Oct 30 19:09:08.085284 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:08.085200 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" podUID="ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" Oct 30 19:09:08.085349 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:09:08.085307 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod48092d6e_157f_4df3_828b_497746965c69.slice/crio-4c2411c26ad8bcc9d0c8af2f555f401cc9d586f3d4d484c63f58a4be5968bd31 WatchSource:0}: Error finding container 4c2411c26ad8bcc9d0c8af2f555f401cc9d586f3d4d484c63f58a4be5968bd31: Status 404 returned error can't find the container with id 4c2411c26ad8bcc9d0c8af2f555f401cc9d586f3d4d484c63f58a4be5968bd31 Oct 30 19:09:08.219906 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:08.219609 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" event={"ID":"48092d6e-157f-4df3-828b-497746965c69","Type":"ContainerStarted","Data":"f2a9008231c412a37ddd6c059aee1b45bcc4c928cc41aac91d4b94b103844b33"} Oct 30 19:09:08.219906 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:08.219665 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" event={"ID":"48092d6e-157f-4df3-828b-497746965c69","Type":"ContainerStarted","Data":"4c2411c26ad8bcc9d0c8af2f555f401cc9d586f3d4d484c63f58a4be5968bd31"} Oct 30 19:09:08.542712 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:08.532753 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:09.085247 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:09.085211 5894 patch_prober.go:29] interesting pod/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body= Oct 30 19:09:09.085419 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:09.085278 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" podUID="ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" Oct 30 19:09:09.580600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:09.576250 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:10.085422 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:10.085393 5894 patch_prober.go:29] interesting pod/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body= Oct 30 19:09:10.085626 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:10.085439 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" podUID="ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" Oct 30 19:09:11.090998 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:11.090965 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:09:11.091413 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:11.091394 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:09:11.092071 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:11.092049 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:09:11.315298 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:11.315246 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" podStartSLOduration=5.315230663 podStartE2EDuration="5.315230663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:09:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:09:08.353052543 +0000 UTC m=+1818.843180363" watchObservedRunningTime="2026-10-30 19:09:11.315230663 +0000 UTC m=+1821.805358483" Oct 30 19:09:11.457259 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:11.457201 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:14.231286 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:14.229975 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:14.693326 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:14.693279 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:14.900322 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:14.900284 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:15.093214 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:15.093187 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9092-0c42f7"] Oct 30 19:09:15.093909 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:15.093893 5894 cpu_manager.go:414] "containerMap: removing container" podUID="7488e3ba-340f-4d02-806d-7b22cc544066" containerName="istio-proxy" Oct 30 19:09:15.093965 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:15.093911 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7488e3ba-340f-4d02-806d-7b22cc544066" containerName="istio-proxy" Oct 30 19:09:15.094035 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:15.094025 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="7488e3ba-340f-4d02-806d-7b22cc544066" containerName="istio-proxy" Oct 30 19:09:15.097659 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:15.097642 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9092-0c42f7" Oct 30 19:09:15.140339 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:15.140302 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shkrv\" (UniqueName: \"kubernetes.io/projected/9234a18b-8146-4871-9eed-0d33749289d0-kube-api-access-shkrv\") pod \"test-curl-9092-0c42f7\" (UID: \"9234a18b-8146-4871-9eed-0d33749289d0\") " pod="opendatahub/test-curl-9092-0c42f7" Oct 30 19:09:15.152010 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:15.151978 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9092-0c42f7"] Oct 30 19:09:15.196794 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:15.196760 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7v9w4\"" Oct 30 19:09:15.242306 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:15.242274 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shkrv\" (UniqueName: \"kubernetes.io/projected/9234a18b-8146-4871-9eed-0d33749289d0-kube-api-access-shkrv\") pod \"test-curl-9092-0c42f7\" (UID: \"9234a18b-8146-4871-9eed-0d33749289d0\") " pod="opendatahub/test-curl-9092-0c42f7" Oct 30 19:09:15.300420 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:15.300388 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-shkrv\" (UniqueName: \"kubernetes.io/projected/9234a18b-8146-4871-9eed-0d33749289d0-kube-api-access-shkrv\") pod \"test-curl-9092-0c42f7\" (UID: \"9234a18b-8146-4871-9eed-0d33749289d0\") " pod="opendatahub/test-curl-9092-0c42f7" Oct 30 19:09:15.411472 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:15.411443 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9092-0c42f7" Oct 30 19:09:15.886600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:15.874404 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9092-0c42f7"] Oct 30 19:09:15.886954 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:09:15.874979 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9234a18b_8146_4871_9eed_0d33749289d0.slice/crio-784c75c4913683f95d3eeb1b97d65c485a8a42fb5c8ebf64111c4851a3bab94e WatchSource:0}: Error finding container 784c75c4913683f95d3eeb1b97d65c485a8a42fb5c8ebf64111c4851a3bab94e: Status 404 returned error can't find the container with id 784c75c4913683f95d3eeb1b97d65c485a8a42fb5c8ebf64111c4851a3bab94e Oct 30 19:09:16.343209 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:16.343158 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9092-0c42f7" event={"ID":"9234a18b-8146-4871-9eed-0d33749289d0","Type":"ContainerStarted","Data":"074d9e72e047994cc7f9be38f91c87d01b4119d1152de0a267ffd3e7bdf04702"} Oct 30 19:09:16.343793 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:16.343769 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9092-0c42f7" event={"ID":"9234a18b-8146-4871-9eed-0d33749289d0","Type":"ContainerStarted","Data":"784c75c4913683f95d3eeb1b97d65c485a8a42fb5c8ebf64111c4851a3bab94e"} Oct 30 19:09:16.462125 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:16.462066 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-9092-0c42f7" podStartSLOduration=1.462050934 podStartE2EDuration="1.462050934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:09:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:09:16.458418755 +0000 UTC m=+1826.948546576" watchObservedRunningTime="2026-10-30 19:09:16.462050934 +0000 UTC m=+1826.952178754" Oct 30 19:09:17.531296 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:17.531261 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:17.544807 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:17.544773 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:17.944223 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:17.944183 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9092-0c42f7"] Oct 30 19:09:17.951739 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:17.951708 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9092-0c42f7"] Oct 30 19:09:18.115708 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:18.115677 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9092-209122"] Oct 30 19:09:18.116442 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:18.116424 5894 cpu_manager.go:414] "containerMap: removing container" podUID="9234a18b-8146-4871-9eed-0d33749289d0" containerName="test-curl-9092-0c42f7" Oct 30 19:09:18.116442 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:18.116443 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9234a18b-8146-4871-9eed-0d33749289d0" containerName="test-curl-9092-0c42f7" Oct 30 19:09:18.116608 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:18.116569 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="9234a18b-8146-4871-9eed-0d33749289d0" containerName="test-curl-9092-0c42f7" Oct 30 19:09:18.120262 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:18.120244 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9092-209122" Oct 30 19:09:18.174303 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:18.174271 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9092-209122"] Oct 30 19:09:18.226550 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:18.226447 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dpcpj\" (UniqueName: \"kubernetes.io/projected/85bbb559-4842-40ac-be9f-d065bf5b54de-kube-api-access-dpcpj\") pod \"test-curl-9092-209122\" (UID: \"85bbb559-4842-40ac-be9f-d065bf5b54de\") " pod="opendatahub/test-curl-9092-209122" Oct 30 19:09:18.328349 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:18.328312 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dpcpj\" (UniqueName: \"kubernetes.io/projected/85bbb559-4842-40ac-be9f-d065bf5b54de-kube-api-access-dpcpj\") pod \"test-curl-9092-209122\" (UID: \"85bbb559-4842-40ac-be9f-d065bf5b54de\") " pod="opendatahub/test-curl-9092-209122" Oct 30 19:09:18.355920 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:18.355876 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-9092-0c42f7" podUID="9234a18b-8146-4871-9eed-0d33749289d0" containerName="test-curl-9092-0c42f7" containerID="cri-o://074d9e72e047994cc7f9be38f91c87d01b4119d1152de0a267ffd3e7bdf04702" gracePeriod=2 Oct 30 19:09:18.368785 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:18.368757 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:18.386234 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:18.386201 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dpcpj\" (UniqueName: \"kubernetes.io/projected/85bbb559-4842-40ac-be9f-d065bf5b54de-kube-api-access-dpcpj\") pod \"test-curl-9092-209122\" (UID: \"85bbb559-4842-40ac-be9f-d065bf5b54de\") " pod="opendatahub/test-curl-9092-209122" Oct 30 19:09:18.407362 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:18.407321 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9092-0c42f7\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="9234a18b-8146-4871-9eed-0d33749289d0" pod="opendatahub/test-curl-9092-0c42f7" Oct 30 19:09:18.434124 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:18.434096 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9092-209122" Oct 30 19:09:18.458879 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:18.458844 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9092-0c42f7\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="9234a18b-8146-4871-9eed-0d33749289d0" pod="opendatahub/test-curl-9092-0c42f7" Oct 30 19:09:18.817177 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:18.817146 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9092-209122"] Oct 30 19:09:18.818368 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:09:18.818343 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod85bbb559_4842_40ac_be9f_d065bf5b54de.slice/crio-6188d7738609edd9db584ddf9cf1289bf07d71db34f94ade0f12421c8954f83a WatchSource:0}: Error finding container 6188d7738609edd9db584ddf9cf1289bf07d71db34f94ade0f12421c8954f83a: Status 404 returned error can't find the container with id 6188d7738609edd9db584ddf9cf1289bf07d71db34f94ade0f12421c8954f83a Oct 30 19:09:18.947793 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:18.947744 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:19.367662 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:19.367315 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9092-209122" event={"ID":"85bbb559-4842-40ac-be9f-d065bf5b54de","Type":"ContainerStarted","Data":"50cec2dd09355ef963a7bac0877b8fa5d58c7b6c1f2600e81dcfd3f731f6b1d6"} Oct 30 19:09:19.367662 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:19.367369 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9092-209122" event={"ID":"85bbb559-4842-40ac-be9f-d065bf5b54de","Type":"ContainerStarted","Data":"6188d7738609edd9db584ddf9cf1289bf07d71db34f94ade0f12421c8954f83a"} Oct 30 19:09:19.418769 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:19.418719 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9092-0c42f7\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="9234a18b-8146-4871-9eed-0d33749289d0" pod="opendatahub/test-curl-9092-0c42f7" Oct 30 19:09:19.542908 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:19.542809 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-9092-209122" podStartSLOduration=1.542794528 podStartE2EDuration="1.542794528s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:09:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:09:19.538862122 +0000 UTC m=+1830.028989943" watchObservedRunningTime="2026-10-30 19:09:19.542794528 +0000 UTC m=+1830.032922347" Oct 30 19:09:19.579656 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:19.579620 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body= Oct 30 19:09:19.579821 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:19.579680 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" podUID="2a4529e2-c317-4d6d-916f-b77baf48009a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" Oct 30 19:09:19.698249 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:19.698206 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9092-0c42f7\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="9234a18b-8146-4871-9eed-0d33749289d0" pod="opendatahub/test-curl-9092-0c42f7" Oct 30 19:09:20.055772 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:20.055516 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:20.155429 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:20.155393 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:09:20.375965 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:20.375940 5894 generic.go:356] "Generic (PLEG): container finished" podID="2a4529e2-c317-4d6d-916f-b77baf48009a" containerID="6f68a8b0b7791d38ab0ecdac757da4175e5501417c6265b6ef12dd5c03881a00" exitCode=0 Oct 30 19:09:20.376064 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:20.376011 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" event={"ID":"2a4529e2-c317-4d6d-916f-b77baf48009a","Type":"ContainerDied","Data":"6f68a8b0b7791d38ab0ecdac757da4175e5501417c6265b6ef12dd5c03881a00"} Oct 30 19:09:20.376467 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:20.376445 5894 scope.go:122] "RemoveContainer" containerID="6f68a8b0b7791d38ab0ecdac757da4175e5501417c6265b6ef12dd5c03881a00" Oct 30 19:09:20.378208 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:20.378177 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:09:20.378287 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:20.378269 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:09:20.378340 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:20.378325 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:09:20.683070 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:20.683039 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9092-0c42f7" Oct 30 19:09:20.781514 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:20.781481 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9234a18b-8146-4871-9eed-0d33749289d0-kube-api-access-shkrv\" (UniqueName: \"kubernetes.io/projected/9234a18b-8146-4871-9eed-0d33749289d0-kube-api-access-shkrv\") pod \"9234a18b-8146-4871-9eed-0d33749289d0\" (UID: \"9234a18b-8146-4871-9eed-0d33749289d0\") " Oct 30 19:09:20.783631 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:20.783600 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9234a18b-8146-4871-9eed-0d33749289d0-kube-api-access-shkrv" pod "9234a18b-8146-4871-9eed-0d33749289d0" (UID: "9234a18b-8146-4871-9eed-0d33749289d0"). InnerVolumeSpecName "kube-api-access-shkrv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:09:20.884507 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:20.884422 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-shkrv\" (UniqueName: \"kubernetes.io/projected/9234a18b-8146-4871-9eed-0d33749289d0-kube-api-access-shkrv\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:09:21.076623 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.076531 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9092-209122"] Oct 30 19:09:21.081266 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.081237 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9092-209122"] Oct 30 19:09:21.255531 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.255492 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9092-628608"] Oct 30 19:09:21.256289 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.256267 5894 cpu_manager.go:414] "containerMap: removing container" podUID="85bbb559-4842-40ac-be9f-d065bf5b54de" containerName="test-curl-9092-209122" Oct 30 19:09:21.256389 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.256289 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="85bbb559-4842-40ac-be9f-d065bf5b54de" containerName="test-curl-9092-209122" Oct 30 19:09:21.256502 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.256488 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="85bbb559-4842-40ac-be9f-d065bf5b54de" containerName="test-curl-9092-209122" Oct 30 19:09:21.260113 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.260091 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9092-628608" Oct 30 19:09:21.295841 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.295817 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9092-628608"] Oct 30 19:09:21.384746 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.384714 5894 generic.go:356] "Generic (PLEG): container finished" podID="9234a18b-8146-4871-9eed-0d33749289d0" containerID="074d9e72e047994cc7f9be38f91c87d01b4119d1152de0a267ffd3e7bdf04702" exitCode=137 Oct 30 19:09:21.384746 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.384754 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9092-0c42f7" Oct 30 19:09:21.385014 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.384817 5894 scope.go:122] "RemoveContainer" containerID="074d9e72e047994cc7f9be38f91c87d01b4119d1152de0a267ffd3e7bdf04702" Oct 30 19:09:21.387252 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.387225 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" event={"ID":"2a4529e2-c317-4d6d-916f-b77baf48009a","Type":"ContainerStarted","Data":"6f4b2b64a18baada6b2d8bad477a01dc946cab7fdbb9b800c3981c9746823b87"} Oct 30 19:09:21.387391 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.387309 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-9092-209122" podUID="85bbb559-4842-40ac-be9f-d065bf5b54de" containerName="test-curl-9092-209122" containerID="cri-o://50cec2dd09355ef963a7bac0877b8fa5d58c7b6c1f2600e81dcfd3f731f6b1d6" gracePeriod=2 Oct 30 19:09:21.392961 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.392933 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4s4kl\" (UniqueName: \"kubernetes.io/projected/fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f-kube-api-access-4s4kl\") pod \"test-curl-9092-628608\" (UID: \"fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f\") " pod="opendatahub/test-curl-9092-628608" Oct 30 19:09:21.396046 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.396021 5894 scope.go:122] "RemoveContainer" containerID="074d9e72e047994cc7f9be38f91c87d01b4119d1152de0a267ffd3e7bdf04702" Oct 30 19:09:21.396334 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:21.396316 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"074d9e72e047994cc7f9be38f91c87d01b4119d1152de0a267ffd3e7bdf04702\": container with ID starting with 074d9e72e047994cc7f9be38f91c87d01b4119d1152de0a267ffd3e7bdf04702 not found: ID does not exist" containerID="074d9e72e047994cc7f9be38f91c87d01b4119d1152de0a267ffd3e7bdf04702" Oct 30 19:09:21.396417 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.396343 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"074d9e72e047994cc7f9be38f91c87d01b4119d1152de0a267ffd3e7bdf04702"} err="failed to get container status \"074d9e72e047994cc7f9be38f91c87d01b4119d1152de0a267ffd3e7bdf04702\": rpc error: code = NotFound desc = could not find container \"074d9e72e047994cc7f9be38f91c87d01b4119d1152de0a267ffd3e7bdf04702\": container with ID starting with 074d9e72e047994cc7f9be38f91c87d01b4119d1152de0a267ffd3e7bdf04702 not found: ID does not exist" Oct 30 19:09:21.451904 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:21.451838 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9092-209122\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="85bbb559-4842-40ac-be9f-d065bf5b54de" pod="opendatahub/test-curl-9092-209122" Oct 30 19:09:21.494542 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.494507 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4s4kl\" (UniqueName: \"kubernetes.io/projected/fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f-kube-api-access-4s4kl\") pod \"test-curl-9092-628608\" (UID: \"fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f\") " pod="opendatahub/test-curl-9092-628608" Oct 30 19:09:21.554629 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.554534 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4s4kl\" (UniqueName: \"kubernetes.io/projected/fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f-kube-api-access-4s4kl\") pod \"test-curl-9092-628608\" (UID: \"fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f\") " pod="opendatahub/test-curl-9092-628608" Oct 30 19:09:21.573781 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.573750 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9092-628608" Oct 30 19:09:21.583455 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.583430 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:09:21.583686 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.583665 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body= Oct 30 19:09:21.583751 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.583711 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" podUID="2a4529e2-c317-4d6d-916f-b77baf48009a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" Oct 30 19:09:21.626210 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:21.625899 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9092-0c42f7\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="9234a18b-8146-4871-9eed-0d33749289d0" pod="opendatahub/test-curl-9092-0c42f7" Oct 30 19:09:21.673527 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.673489 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9234a18b-8146-4871-9eed-0d33749289d0" path="/var/lib/kubelet/pods/9234a18b-8146-4871-9eed-0d33749289d0/volumes" Oct 30 19:09:21.677344 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:21.677308 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9092-209122\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="85bbb559-4842-40ac-be9f-d065bf5b54de" pod="opendatahub/test-curl-9092-209122" Oct 30 19:09:21.975886 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:09:21.975857 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc77d31f_ec60_4a7e_8cf8_89d8872a0b5f.slice/crio-742ef19ca2687ce603fd1aad101fd341e7f8bc60783e914117a8ab21660de2cb WatchSource:0}: Error finding container 742ef19ca2687ce603fd1aad101fd341e7f8bc60783e914117a8ab21660de2cb: Status 404 returned error can't find the container with id 742ef19ca2687ce603fd1aad101fd341e7f8bc60783e914117a8ab21660de2cb Oct 30 19:09:21.976066 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:21.976045 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9092-628608"] Oct 30 19:09:22.305020 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:22.304934 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:22.395516 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:22.395474 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9092-628608" event={"ID":"fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f","Type":"ContainerStarted","Data":"f57e91526f5acae1e20e71df44eb1c3212a462cd122ee6d397176bccff344bf7"} Oct 30 19:09:22.395721 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:22.395523 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9092-628608" event={"ID":"fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f","Type":"ContainerStarted","Data":"742ef19ca2687ce603fd1aad101fd341e7f8bc60783e914117a8ab21660de2cb"} Oct 30 19:09:22.513791 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:22.513741 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-9092-628608" podStartSLOduration=1.5137270470000002 podStartE2EDuration="1.513727047s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:09:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:09:22.512024156 +0000 UTC m=+1833.002151976" watchObservedRunningTime="2026-10-30 19:09:22.513727047 +0000 UTC m=+1833.003854867" Oct 30 19:09:22.547116 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:22.547050 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:22.584491 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:22.584424 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body= Oct 30 19:09:22.584491 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:22.584483 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" podUID="2a4529e2-c317-4d6d-916f-b77baf48009a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" Oct 30 19:09:23.408176 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:23.408148 5894 generic.go:356] "Generic (PLEG): container finished" podID="85bbb559-4842-40ac-be9f-d065bf5b54de" containerID="50cec2dd09355ef963a7bac0877b8fa5d58c7b6c1f2600e81dcfd3f731f6b1d6" exitCode=137 Oct 30 19:09:23.583876 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:23.583846 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body= Oct 30 19:09:23.584046 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:23.583892 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" podUID="2a4529e2-c317-4d6d-916f-b77baf48009a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" Oct 30 19:09:23.690617 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:23.690589 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9092-209122" Oct 30 19:09:23.854539 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:23.854489 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/85bbb559-4842-40ac-be9f-d065bf5b54de-kube-api-access-dpcpj\" (UniqueName: \"kubernetes.io/projected/85bbb559-4842-40ac-be9f-d065bf5b54de-kube-api-access-dpcpj\") pod \"85bbb559-4842-40ac-be9f-d065bf5b54de\" (UID: \"85bbb559-4842-40ac-be9f-d065bf5b54de\") " Oct 30 19:09:23.856980 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:23.856947 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/85bbb559-4842-40ac-be9f-d065bf5b54de-kube-api-access-dpcpj" pod "85bbb559-4842-40ac-be9f-d065bf5b54de" (UID: "85bbb559-4842-40ac-be9f-d065bf5b54de"). InnerVolumeSpecName "kube-api-access-dpcpj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:09:23.956821 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:23.956741 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dpcpj\" (UniqueName: \"kubernetes.io/projected/85bbb559-4842-40ac-be9f-d065bf5b54de-kube-api-access-dpcpj\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:09:24.014755 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:24.014720 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9092-628608"] Oct 30 19:09:24.058975 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:24.058938 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9092-628608"] Oct 30 19:09:24.190972 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:24.190932 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9092-559793"] Oct 30 19:09:24.191693 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:24.191672 5894 cpu_manager.go:414] "containerMap: removing container" podUID="fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f" containerName="test-curl-9092-628608" Oct 30 19:09:24.191808 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:24.191692 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f" containerName="test-curl-9092-628608" Oct 30 19:09:24.191894 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:24.191879 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f" containerName="test-curl-9092-628608" Oct 30 19:09:24.195475 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:24.195454 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9092-559793" Oct 30 19:09:24.257064 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:24.256973 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9092-559793"] Oct 30 19:09:24.363587 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:24.363547 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6wtsx\" (UniqueName: \"kubernetes.io/projected/7c84b026-90ca-46c7-90ea-16f3fdc09bbb-kube-api-access-6wtsx\") pod \"test-curl-9092-559793\" (UID: \"7c84b026-90ca-46c7-90ea-16f3fdc09bbb\") " pod="opendatahub/test-curl-9092-559793" Oct 30 19:09:24.419454 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:24.419415 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-9092-628608" podUID="fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f" containerName="test-curl-9092-628608" containerID="cri-o://f57e91526f5acae1e20e71df44eb1c3212a462cd122ee6d397176bccff344bf7" gracePeriod=2 Oct 30 19:09:24.419920 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:24.419788 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9092-209122" Oct 30 19:09:24.422724 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:24.422697 5894 scope.go:122] "RemoveContainer" containerID="50cec2dd09355ef963a7bac0877b8fa5d58c7b6c1f2600e81dcfd3f731f6b1d6" Oct 30 19:09:24.464677 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:24.464649 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6wtsx\" (UniqueName: \"kubernetes.io/projected/7c84b026-90ca-46c7-90ea-16f3fdc09bbb-kube-api-access-6wtsx\") pod \"test-curl-9092-559793\" (UID: \"7c84b026-90ca-46c7-90ea-16f3fdc09bbb\") " pod="opendatahub/test-curl-9092-559793" Oct 30 19:09:24.474058 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:24.474028 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9092-628608\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f" pod="opendatahub/test-curl-9092-628608" Oct 30 19:09:24.523258 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:24.523205 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6wtsx\" (UniqueName: \"kubernetes.io/projected/7c84b026-90ca-46c7-90ea-16f3fdc09bbb-kube-api-access-6wtsx\") pod \"test-curl-9092-559793\" (UID: \"7c84b026-90ca-46c7-90ea-16f3fdc09bbb\") " pod="opendatahub/test-curl-9092-559793" Oct 30 19:09:24.524970 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:24.524943 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9092-209122\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="85bbb559-4842-40ac-be9f-d065bf5b54de" pod="opendatahub/test-curl-9092-209122" Oct 30 19:09:24.576838 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:24.576794 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9092-628608\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f" pod="opendatahub/test-curl-9092-628608" Oct 30 19:09:24.583665 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:24.583630 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body= Oct 30 19:09:24.583784 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:24.583694 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" podUID="2a4529e2-c317-4d6d-916f-b77baf48009a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" Oct 30 19:09:24.809051 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:24.808967 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9092-559793" Oct 30 19:09:24.902597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:24.900948 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:25.396927 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:25.396899 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9092-559793"] Oct 30 19:09:25.397556 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:09:25.397523 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7c84b026_90ca_46c7_90ea_16f3fdc09bbb.slice/crio-c2962fcaf0da46b29e98922c88e33773f942590226d68bed4cada91266ec2abe WatchSource:0}: Error finding container c2962fcaf0da46b29e98922c88e33773f942590226d68bed4cada91266ec2abe: Status 404 returned error can't find the container with id c2962fcaf0da46b29e98922c88e33773f942590226d68bed4cada91266ec2abe Oct 30 19:09:25.461163 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:25.461126 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9092-559793" event={"ID":"7c84b026-90ca-46c7-90ea-16f3fdc09bbb","Type":"ContainerStarted","Data":"c2962fcaf0da46b29e98922c88e33773f942590226d68bed4cada91266ec2abe"} Oct 30 19:09:25.583690 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:25.583647 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body= Oct 30 19:09:25.583882 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:25.583703 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" podUID="2a4529e2-c317-4d6d-916f-b77baf48009a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" Oct 30 19:09:25.660115 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:25.660030 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="85bbb559-4842-40ac-be9f-d065bf5b54de" path="/var/lib/kubelet/pods/85bbb559-4842-40ac-be9f-d065bf5b54de/volumes" Oct 30 19:09:26.470777 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:26.470741 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9092-559793" event={"ID":"7c84b026-90ca-46c7-90ea-16f3fdc09bbb","Type":"ContainerStarted","Data":"e4789c225d9ec33dda78d2af45aab6a79260b1bf620372a9dae604743e54324d"} Oct 30 19:09:26.583984 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:26.583946 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" start-of-body= Oct 30 19:09:26.584145 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:26.584030 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" podUID="2a4529e2-c317-4d6d-916f-b77baf48009a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.98:15021/healthz/ready\": dial tcp 10.132.0.98:15021: connect: connection refused" Oct 30 19:09:26.594733 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:26.594673 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-9092-559793" podStartSLOduration=2.59465433 podStartE2EDuration="2.59465433s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:09:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:09:26.588264768 +0000 UTC m=+1837.078392589" watchObservedRunningTime="2026-10-30 19:09:26.59465433 +0000 UTC m=+1837.084782153" Oct 30 19:09:26.731538 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:26.731515 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9092-628608" Oct 30 19:09:26.829208 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:26.829177 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f-kube-api-access-4s4kl\" (UniqueName: \"kubernetes.io/projected/fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f-kube-api-access-4s4kl\") pod \"fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f\" (UID: \"fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f\") " Oct 30 19:09:26.831446 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:26.831420 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f-kube-api-access-4s4kl" pod "fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f" (UID: "fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f"). InnerVolumeSpecName "kube-api-access-4s4kl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:09:26.931348 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:26.931317 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4s4kl\" (UniqueName: \"kubernetes.io/projected/fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f-kube-api-access-4s4kl\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:09:27.478515 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:27.478475 5894 generic.go:356] "Generic (PLEG): container finished" podID="fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f" containerID="f57e91526f5acae1e20e71df44eb1c3212a462cd122ee6d397176bccff344bf7" exitCode=137 Oct 30 19:09:27.478515 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:27.478516 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9092-628608" Oct 30 19:09:27.479019 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:27.478617 5894 scope.go:122] "RemoveContainer" containerID="f57e91526f5acae1e20e71df44eb1c3212a462cd122ee6d397176bccff344bf7" Oct 30 19:09:27.488252 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:27.488236 5894 scope.go:122] "RemoveContainer" containerID="f57e91526f5acae1e20e71df44eb1c3212a462cd122ee6d397176bccff344bf7" Oct 30 19:09:27.488505 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:27.488483 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f57e91526f5acae1e20e71df44eb1c3212a462cd122ee6d397176bccff344bf7\": container with ID starting with f57e91526f5acae1e20e71df44eb1c3212a462cd122ee6d397176bccff344bf7 not found: ID does not exist" containerID="f57e91526f5acae1e20e71df44eb1c3212a462cd122ee6d397176bccff344bf7" Oct 30 19:09:27.488601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:27.488517 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f57e91526f5acae1e20e71df44eb1c3212a462cd122ee6d397176bccff344bf7"} err="failed to get container status \"f57e91526f5acae1e20e71df44eb1c3212a462cd122ee6d397176bccff344bf7\": rpc error: code = NotFound desc = could not find container \"f57e91526f5acae1e20e71df44eb1c3212a462cd122ee6d397176bccff344bf7\": container with ID starting with f57e91526f5acae1e20e71df44eb1c3212a462cd122ee6d397176bccff344bf7 not found: ID does not exist" Oct 30 19:09:27.538083 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:27.538051 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9092-628608\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f" pod="opendatahub/test-curl-9092-628608" Oct 30 19:09:27.589474 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:27.589448 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:09:27.589925 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:27.589901 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:09:27.590562 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:27.590543 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:09:27.651347 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:27.651314 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f" path="/var/lib/kubelet/pods/fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f/volumes" Oct 30 19:09:27.807270 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:27.807180 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9092-628608\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="fc77d31f-ec60-4a7e-8cf8-89d8872a0b5f" pod="opendatahub/test-curl-9092-628608" Oct 30 19:09:28.101983 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:28.101905 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9092-559793"] Oct 30 19:09:28.106393 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:28.106360 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9092-559793"] Oct 30 19:09:28.486893 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:28.486829 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-9092-559793" podUID="7c84b026-90ca-46c7-90ea-16f3fdc09bbb" containerName="test-curl-9092-559793" containerID="cri-o://e4789c225d9ec33dda78d2af45aab6a79260b1bf620372a9dae604743e54324d" gracePeriod=2 Oct 30 19:09:28.538080 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:28.538048 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9092-559793\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="7c84b026-90ca-46c7-90ea-16f3fdc09bbb" pod="opendatahub/test-curl-9092-559793" Oct 30 19:09:28.673916 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:28.673880 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:29.478596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:29.458887 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:29.789424 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:29.784070 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9092-559793\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="7c84b026-90ca-46c7-90ea-16f3fdc09bbb" pod="opendatahub/test-curl-9092-559793" Oct 30 19:09:29.995422 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:29.994138 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:30.304077 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:30.304044 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:30.748777 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:30.748752 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9092-559793" Oct 30 19:09:30.807524 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:30.807490 5894 generic.go:356] "Generic (PLEG): container finished" podID="7c84b026-90ca-46c7-90ea-16f3fdc09bbb" containerID="e4789c225d9ec33dda78d2af45aab6a79260b1bf620372a9dae604743e54324d" exitCode=137 Oct 30 19:09:30.807965 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:30.807547 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9092-559793" Oct 30 19:09:30.807965 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:30.807649 5894 scope.go:122] "RemoveContainer" containerID="e4789c225d9ec33dda78d2af45aab6a79260b1bf620372a9dae604743e54324d" Oct 30 19:09:30.819060 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:30.819032 5894 scope.go:122] "RemoveContainer" containerID="e4789c225d9ec33dda78d2af45aab6a79260b1bf620372a9dae604743e54324d" Oct 30 19:09:30.819350 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:30.819329 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4789c225d9ec33dda78d2af45aab6a79260b1bf620372a9dae604743e54324d\": container with ID starting with e4789c225d9ec33dda78d2af45aab6a79260b1bf620372a9dae604743e54324d not found: ID does not exist" containerID="e4789c225d9ec33dda78d2af45aab6a79260b1bf620372a9dae604743e54324d" Oct 30 19:09:30.819436 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:30.819360 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4789c225d9ec33dda78d2af45aab6a79260b1bf620372a9dae604743e54324d"} err="failed to get container status \"e4789c225d9ec33dda78d2af45aab6a79260b1bf620372a9dae604743e54324d\": rpc error: code = NotFound desc = could not find container \"e4789c225d9ec33dda78d2af45aab6a79260b1bf620372a9dae604743e54324d\": container with ID starting with e4789c225d9ec33dda78d2af45aab6a79260b1bf620372a9dae604743e54324d not found: ID does not exist" Oct 30 19:09:30.870272 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:30.870190 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7c84b026-90ca-46c7-90ea-16f3fdc09bbb-kube-api-access-6wtsx\" (UniqueName: \"kubernetes.io/projected/7c84b026-90ca-46c7-90ea-16f3fdc09bbb-kube-api-access-6wtsx\") pod \"7c84b026-90ca-46c7-90ea-16f3fdc09bbb\" (UID: \"7c84b026-90ca-46c7-90ea-16f3fdc09bbb\") " Oct 30 19:09:30.872364 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:30.872337 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c84b026-90ca-46c7-90ea-16f3fdc09bbb-kube-api-access-6wtsx" pod "7c84b026-90ca-46c7-90ea-16f3fdc09bbb" (UID: "7c84b026-90ca-46c7-90ea-16f3fdc09bbb"). InnerVolumeSpecName "kube-api-access-6wtsx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:09:30.988482 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:30.988436 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6wtsx\" (UniqueName: \"kubernetes.io/projected/7c84b026-90ca-46c7-90ea-16f3fdc09bbb-kube-api-access-6wtsx\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:09:31.168430 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:31.168391 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9092-559793\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="7c84b026-90ca-46c7-90ea-16f3fdc09bbb" pod="opendatahub/test-curl-9092-559793" Oct 30 19:09:31.652458 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:31.652413 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7c84b026-90ca-46c7-90ea-16f3fdc09bbb" path="/var/lib/kubelet/pods/7c84b026-90ca-46c7-90ea-16f3fdc09bbb/volumes" Oct 30 19:09:33.650936 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:33.650899 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:33.890883 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:33.890850 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:35.629644 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:35.617497 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:37.300632 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:37.273367 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:37.682290 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:37.682250 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": context deadline exceeded" Oct 30 19:09:37.709523 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:37.706325 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:37.719662 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:37.719616 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded" Oct 30 19:09:38.302816 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:38.302747 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" podUID="ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:09:38.338399 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:38.338356 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:09:38.343441 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:38.343387 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:09:40.285714 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:40.285661 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" podUID="ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:09:40.322721 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:40.322666 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:09:40.329696 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:40.329655 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:09:40.394602 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:40.394540 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" podUID="48092d6e-157f-4df3-828b-497746965c69" containerName="main" probeResult="failure" output="Get \"https://10.132.0.105:8000/ready\": context deadline exceeded" Oct 30 19:09:40.446823 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:40.446772 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:09:40.680602 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:40.673682 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:09:40.680602 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:40.674053 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded" Oct 30 19:09:40.700805 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:40.700752 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": context deadline exceeded" Oct 30 19:09:40.701462 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:40.701423 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": context deadline exceeded" Oct 30 19:09:41.286822 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:41.286780 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" podUID="ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" containerName="main" probeResult="failure" output="Get \"https://10.132.0.86:8000/ready\": context deadline exceeded" Oct 30 19:09:41.288116 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:41.288093 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:09:41.325790 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:41.325747 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" probeResult="failure" output="Get \"https://10.132.0.64:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:09:41.326337 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:41.326312 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 19:09:41.337689 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:41.337652 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.55:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:09:41.338228 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:41.338206 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 19:09:41.422422 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:41.422380 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" podUID="48092d6e-157f-4df3-828b-497746965c69" containerName="main" probeResult="failure" output="Get \"https://10.132.0.105:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:09:41.424454 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:41.423474 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:41.475597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:41.456013 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" probeResult="failure" output="Get \"https://10.132.0.54:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:09:41.475597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:41.456405 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 19:09:41.821699 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:41.808923 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" probeResult="failure" output="Get \"https://10.132.0.80:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:09:41.823609 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:41.822487 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" probeResult="failure" output="Get \"https://10.132.0.57:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:09:41.823609 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:41.822605 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/ready\": context deadline exceeded" Oct 30 19:09:41.909596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:41.846648 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" probeResult="failure" output="Get \"https://10.132.0.68:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:09:42.110475 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:42.110361 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:42.134075 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:42.130371 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:09:42.134075 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:42.130443 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 19:09:42.134075 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:42.130478 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:09:42.134075 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:42.130560 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 19:09:42.216905 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:42.216876 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 19:09:42.295747 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:42.294176 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 19:09:42.305179 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:42.305135 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 19:09:42.384894 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:42.383639 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:09:42.384894 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:42.384484 5894 prober.go:122] "Probe failed" probeType="Liveness" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" probeResult="failure" output="Get \"https://10.132.0.59:8000/health\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Oct 30 19:09:42.402594 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:42.392091 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:09:42.480620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:42.480499 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 19:09:42.480620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:42.480562 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 19:09:47.343167 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:47.343129 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:48.926192 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:48.926160 5894 generic.go:356] "Generic (PLEG): container finished" podID="8a237b2f-38a2-47b6-a0a4-3d89b557b419" containerID="398d5698699ffc986b484bf657c390586ca5833ca8ee51e124ca58cc0ff27cc8" exitCode=0 Oct 30 19:09:48.926729 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:48.926233 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" event={"ID":"8a237b2f-38a2-47b6-a0a4-3d89b557b419","Type":"ContainerDied","Data":"398d5698699ffc986b484bf657c390586ca5833ca8ee51e124ca58cc0ff27cc8"} Oct 30 19:09:48.926729 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:48.926720 5894 scope.go:122] "RemoveContainer" containerID="398d5698699ffc986b484bf657c390586ca5833ca8ee51e124ca58cc0ff27cc8" Oct 30 19:09:48.928410 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:48.928382 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:09:48.928497 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:48.928452 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:09:48.928497 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:48.928485 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:09:49.959639 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:49.959562 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" event={"ID":"8a237b2f-38a2-47b6-a0a4-3d89b557b419","Type":"ContainerStarted","Data":"54e1e2ed45877249f1fa42aee46f3072689369d78d35aa7be2a00a0b2dd1bf0d"} Oct 30 19:09:50.118866 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:09:50.118829 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:09:50.125112 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:50.125096 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:09:50.126542 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:50.126517 5894 patch_prober.go:29] interesting pod/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body= Oct 30 19:09:50.126700 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:50.126562 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" podUID="8a237b2f-38a2-47b6-a0a4-3d89b557b419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" Oct 30 19:09:51.125033 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:51.125001 5894 patch_prober.go:29] interesting pod/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body= Oct 30 19:09:51.125413 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:51.125051 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" podUID="8a237b2f-38a2-47b6-a0a4-3d89b557b419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" Oct 30 19:09:52.129885 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:52.129851 5894 patch_prober.go:29] interesting pod/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body= Oct 30 19:09:52.130818 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:52.130787 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" podUID="8a237b2f-38a2-47b6-a0a4-3d89b557b419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" Oct 30 19:09:52.141609 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:52.137922 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:09:53.125001 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:53.124966 5894 patch_prober.go:29] interesting pod/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body= Oct 30 19:09:53.125182 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:53.125018 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" podUID="8a237b2f-38a2-47b6-a0a4-3d89b557b419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" Oct 30 19:09:54.125351 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:54.125321 5894 patch_prober.go:29] interesting pod/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body= Oct 30 19:09:54.125757 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:54.125373 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" podUID="8a237b2f-38a2-47b6-a0a4-3d89b557b419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" Oct 30 19:09:55.124948 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:55.124912 5894 patch_prober.go:29] interesting pod/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" start-of-body= Oct 30 19:09:55.125111 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:55.124971 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" podUID="8a237b2f-38a2-47b6-a0a4-3d89b557b419" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.100:15021/healthz/ready\": dial tcp 10.132.0.100:15021: connect: connection refused" Oct 30 19:09:55.490887 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:55.490810 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:09:56.130079 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:56.130048 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:09:56.131291 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:56.131267 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:09:56.995600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:09:56.994403 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:01.583595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:01.582699 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:01.773128 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:01.773081 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:01.971671 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:01.971633 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:02.997763 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:02.997733 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x"] Oct 30 19:10:02.998405 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:02.998390 5894 cpu_manager.go:414] "containerMap: removing container" podUID="7c84b026-90ca-46c7-90ea-16f3fdc09bbb" containerName="test-curl-9092-559793" Oct 30 19:10:02.998456 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:02.998405 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7c84b026-90ca-46c7-90ea-16f3fdc09bbb" containerName="test-curl-9092-559793" Oct 30 19:10:02.998560 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:02.998550 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="7c84b026-90ca-46c7-90ea-16f3fdc09bbb" containerName="test-curl-9092-559793" Oct 30 19:10:03.003680 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.003662 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.089031 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.088990 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x"] Oct 30 19:10:03.141332 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.141296 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-workload-certs\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.141511 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.141349 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-workload-socket\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.141511 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.141419 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-podinfo\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.141511 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.141447 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-token\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.141511 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.141478 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-credential-socket\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.141672 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.141641 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-data\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.141760 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.141741 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-envoy\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.142059 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.141779 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6db387c-a643-49dd-9ef8-159fc42071d6-istiod-ca-cert\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.142059 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.141925 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l92pj\" (UniqueName: \"kubernetes.io/projected/f6db387c-a643-49dd-9ef8-159fc42071d6-kube-api-access-l92pj\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.243283 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.243248 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-workload-certs\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.243454 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.243305 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-workload-socket\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.243454 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.243337 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-podinfo\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.243454 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.243367 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-token\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.243454 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.243410 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-credential-socket\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.243720 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.243478 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-data\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.243720 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.243540 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-envoy\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.243720 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.243567 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6db387c-a643-49dd-9ef8-159fc42071d6-istiod-ca-cert\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.243877 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.243777 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l92pj\" (UniqueName: \"kubernetes.io/projected/f6db387c-a643-49dd-9ef8-159fc42071d6-kube-api-access-l92pj\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.243877 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.243785 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-workload-certs\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.243877 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.243819 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-workload-socket\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.244055 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.243899 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-credential-socket\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.244055 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.244023 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-data\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.244387 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.244365 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6db387c-a643-49dd-9ef8-159fc42071d6-istiod-ca-cert\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.245853 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.245831 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-envoy\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.245973 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.245956 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-podinfo\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.301076 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.300987 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-token\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.301353 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.301329 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l92pj\" (UniqueName: \"kubernetes.io/projected/f6db387c-a643-49dd-9ef8-159fc42071d6-kube-api-access-l92pj\") pod \"e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.317345 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.317309 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:03.751207 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.751179 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x"] Oct 30 19:10:03.754384 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.754348 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:10:03.754512 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.754414 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:10:03.754512 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:03.754452 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:10:04.064844 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:04.064759 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" event={"ID":"f6db387c-a643-49dd-9ef8-159fc42071d6","Type":"ContainerStarted","Data":"a07df988cb2c03d10d07da80b15d393016f2fd3d8fead4ebd512ec4773d1bb84"} Oct 30 19:10:04.064844 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:04.064815 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" event={"ID":"f6db387c-a643-49dd-9ef8-159fc42071d6","Type":"ContainerStarted","Data":"6a24c36f38ff454d11af0a92c94f80ad95a51c4576afe346fd6d76398d8ac1f8"} Oct 30 19:10:04.189239 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:04.189179 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" podStartSLOduration=2.189159735 podStartE2EDuration="2.189159735s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:10:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:10:04.184256297 +0000 UTC m=+1874.674384119" watchObservedRunningTime="2026-10-30 19:10:04.189159735 +0000 UTC m=+1874.679287558" Oct 30 19:10:04.317973 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:04.317895 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:04.319414 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:04.319387 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body= Oct 30 19:10:04.319539 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:04.319434 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" podUID="f6db387c-a643-49dd-9ef8-159fc42071d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" Oct 30 19:10:05.318088 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:05.318055 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body= Oct 30 19:10:05.318518 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:05.318109 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" podUID="f6db387c-a643-49dd-9ef8-159fc42071d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" Oct 30 19:10:06.318504 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:06.318468 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body= Oct 30 19:10:06.318934 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:06.318516 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" podUID="f6db387c-a643-49dd-9ef8-159fc42071d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" Oct 30 19:10:06.855605 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:06.850702 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:07.318453 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:07.318410 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body= Oct 30 19:10:07.318655 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:07.318466 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" podUID="f6db387c-a643-49dd-9ef8-159fc42071d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" Oct 30 19:10:08.318674 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:08.318639 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body= Oct 30 19:10:08.319076 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:08.318703 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" podUID="f6db387c-a643-49dd-9ef8-159fc42071d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" Oct 30 19:10:09.318372 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:09.318338 5894 patch_prober.go:29] interesting pod/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" start-of-body= Oct 30 19:10:09.318558 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:09.318385 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" podUID="f6db387c-a643-49dd-9ef8-159fc42071d6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.110:15021/healthz/ready\": dial tcp 10.132.0.110:15021: connect: connection refused" Oct 30 19:10:10.181952 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:10.181920 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:10.322881 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:10.322851 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:10.323395 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:10.323377 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:10.324141 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:10.324118 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:10:10.355868 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:10.355843 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:11.534371 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:11.534339 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:11.720338 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:11.720300 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:11.899232 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:11.899196 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:15.926908 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:15.926865 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:17.503758 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:17.503720 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:17.776059 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:17.775971 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:17.968936 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:17.968897 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:18.424278 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:18.424223 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:20.120568 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:10:20.120522 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:10:22.594153 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:22.594107 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:22.975146 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:22.975115 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:24.653320 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:24.653288 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:24.855780 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:24.855750 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:24.999182 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:24.999095 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh"] Oct 30 19:10:24.999393 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:24.999364 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" podUID="ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" containerName="istio-proxy" containerID="cri-o://404b498bc23c0cc8327529f63423b3caaaab0affc8224cc092465328d8890b1d" gracePeriod=30 Oct 30 19:10:25.085561 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:25.085457 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:25.866712 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:25.866669 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:26.094294 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:26.094251 5894 patch_prober.go:29] interesting pod/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:10:26.094469 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:26.094321 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" podUID="ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:10:26.472603 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:26.469104 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:27.086598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:27.081472 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:28.245538 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:28.245501 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:30.302154 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.302129 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:10:30.386176 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.386148 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-kube-api-access-bdpsg\" (UniqueName: \"kubernetes.io/projected/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-kube-api-access-bdpsg\") pod \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " Oct 30 19:10:30.386353 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.386206 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-podinfo\") pod \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " Oct 30 19:10:30.386353 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.386258 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-workload-socket\") pod \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " Oct 30 19:10:30.386353 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.386315 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-workload-certs\") pod \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " Oct 30 19:10:30.386556 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.386424 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-data\") pod \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " Oct 30 19:10:30.386556 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.386501 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-token\" (UniqueName: \"kubernetes.io/projected/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-token\") pod \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " Oct 30 19:10:30.386556 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.386551 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-envoy\") pod \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " Oct 30 19:10:30.386740 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.386598 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-credential-socket\") pod \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " Oct 30 19:10:30.386740 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.386628 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-workload-socket" pod "ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" (UID: "ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:10:30.386740 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.386639 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istiod-ca-cert\") pod \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\" (UID: \"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4\") " Oct 30 19:10:30.386909 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.386745 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-workload-certs" pod "ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" (UID: "ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:10:30.386979 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.386920 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istiod-ca-cert" pod "ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" (UID: "ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:10:30.387343 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.387308 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-credential-socket" pod "ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" (UID: "ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:10:30.387618 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.387558 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-data" pod "ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" (UID: "ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:10:30.387856 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.387838 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:10:30.387954 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.387860 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:10:30.387954 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.387875 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:10:30.387954 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.387889 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:10:30.387954 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.387902 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:10:30.389549 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.389523 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-kube-api-access-bdpsg" pod "ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" (UID: "ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4"). InnerVolumeSpecName "kube-api-access-bdpsg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:10:30.389779 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.389647 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-envoy" pod "ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" (UID: "ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:10:30.390615 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.390118 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-podinfo" pod "ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" (UID: "ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:10:30.390615 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.390455 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-token" pod "ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" (UID: "ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:10:30.396256 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.396230 5894 generic.go:356] "Generic (PLEG): container finished" podID="ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" containerID="404b498bc23c0cc8327529f63423b3caaaab0affc8224cc092465328d8890b1d" exitCode=0 Oct 30 19:10:30.396360 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.396276 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" event={"ID":"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4","Type":"ContainerDied","Data":"404b498bc23c0cc8327529f63423b3caaaab0affc8224cc092465328d8890b1d"} Oct 30 19:10:30.396360 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.396290 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" Oct 30 19:10:30.396360 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.396332 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh" event={"ID":"ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4","Type":"ContainerDied","Data":"12f4d63df275f0685076f21b06965dd923e77427452e9ed94362eb4376cc9ac6"} Oct 30 19:10:30.396517 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.396365 5894 scope.go:122] "RemoveContainer" containerID="404b498bc23c0cc8327529f63423b3caaaab0affc8224cc092465328d8890b1d" Oct 30 19:10:30.425400 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.425381 5894 scope.go:122] "RemoveContainer" containerID="2ef858e03d4bb08b4e636b00ce8def883e42d62a73c340fbcd1ecd0236b88406" Oct 30 19:10:30.435034 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.435013 5894 scope.go:122] "RemoveContainer" containerID="404b498bc23c0cc8327529f63423b3caaaab0affc8224cc092465328d8890b1d" Oct 30 19:10:30.435338 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:10:30.435307 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"404b498bc23c0cc8327529f63423b3caaaab0affc8224cc092465328d8890b1d\": container with ID starting with 404b498bc23c0cc8327529f63423b3caaaab0affc8224cc092465328d8890b1d not found: ID does not exist" containerID="404b498bc23c0cc8327529f63423b3caaaab0affc8224cc092465328d8890b1d" Oct 30 19:10:30.435399 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.435345 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"404b498bc23c0cc8327529f63423b3caaaab0affc8224cc092465328d8890b1d"} err="failed to get container status \"404b498bc23c0cc8327529f63423b3caaaab0affc8224cc092465328d8890b1d\": rpc error: code = NotFound desc = could not find container \"404b498bc23c0cc8327529f63423b3caaaab0affc8224cc092465328d8890b1d\": container with ID starting with 404b498bc23c0cc8327529f63423b3caaaab0affc8224cc092465328d8890b1d not found: ID does not exist" Oct 30 19:10:30.435399 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.435369 5894 scope.go:122] "RemoveContainer" containerID="2ef858e03d4bb08b4e636b00ce8def883e42d62a73c340fbcd1ecd0236b88406" Oct 30 19:10:30.435724 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:10:30.435693 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ef858e03d4bb08b4e636b00ce8def883e42d62a73c340fbcd1ecd0236b88406\": container with ID starting with 2ef858e03d4bb08b4e636b00ce8def883e42d62a73c340fbcd1ecd0236b88406 not found: ID does not exist" containerID="2ef858e03d4bb08b4e636b00ce8def883e42d62a73c340fbcd1ecd0236b88406" Oct 30 19:10:30.435778 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.435722 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ef858e03d4bb08b4e636b00ce8def883e42d62a73c340fbcd1ecd0236b88406"} err="failed to get container status \"2ef858e03d4bb08b4e636b00ce8def883e42d62a73c340fbcd1ecd0236b88406\": rpc error: code = NotFound desc = could not find container \"2ef858e03d4bb08b4e636b00ce8def883e42d62a73c340fbcd1ecd0236b88406\": container with ID starting with 2ef858e03d4bb08b4e636b00ce8def883e42d62a73c340fbcd1ecd0236b88406 not found: ID does not exist" Oct 30 19:10:30.490631 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.490602 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:10:30.490631 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.490633 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bdpsg\" (UniqueName: \"kubernetes.io/projected/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-kube-api-access-bdpsg\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:10:30.490631 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.490652 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:10:30.491032 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.490663 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:10:30.581849 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.581692 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh"] Oct 30 19:10:30.585441 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.585416 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-4501c8d4-openshift-default-79f68c9489-q7srh"] Oct 30 19:10:30.943631 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:30.943568 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:31.148946 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:31.148874 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:31.485599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:31.485542 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:31.651780 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:31.651748 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" path="/var/lib/kubelet/pods/ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4/volumes" Oct 30 19:10:32.317507 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:32.317477 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:33.043660 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:33.033953 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:33.711597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:33.708520 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:34.078647 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:34.078014 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:34.260207 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:34.260176 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:34.637010 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:34.636969 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:37.940674 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:37.940641 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:38.188072 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:38.188042 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:38.393424 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:38.393395 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:39.708599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:39.708070 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:41.582624 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:41.582558 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:41.772427 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:41.772390 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:41.971998 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:41.971964 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:42.351712 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:42.333758 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:44.977014 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:44.976984 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:50.104405 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:50.104373 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:50.119447 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:10:50.119418 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:10:52.437409 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:52.437373 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:52.729941 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:52.729906 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:54.496148 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:54.496117 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df"] Oct 30 19:10:54.496648 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:54.496389 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" podUID="293b1916-b37f-46c7-8ce6-b9efa6a50809" containerName="istio-proxy" containerID="cri-o://0c03b664de0e6d627b50ffbf3913a252a173cf13b971f1b49bd26476f1c1bc09" gracePeriod=30 Oct 30 19:10:54.831551 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:54.831463 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:10:54.831551 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:54.831528 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" podUID="293b1916-b37f-46c7-8ce6-b9efa6a50809" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:10:54.949937 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:54.949899 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr"] Oct 30 19:10:54.950189 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:54.950144 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" podUID="0133d032-d0fb-4623-8c0e-4ddc8b765c6d" containerName="istio-proxy" containerID="cri-o://5941388e9c1d6ca368b3b98ca036b6bdf8980a374e924171797f95fe1850d341" gracePeriod=30 Oct 30 19:10:55.300599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:55.289644 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:10:55.913618 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:55.913564 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:10:55.913997 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:55.913631 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" podUID="0133d032-d0fb-4623-8c0e-4ddc8b765c6d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:10:59.778825 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.778766 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 19:10:59.863553 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.863521 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-envoy\") pod \"293b1916-b37f-46c7-8ce6-b9efa6a50809\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " Oct 30 19:10:59.863750 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.863696 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/293b1916-b37f-46c7-8ce6-b9efa6a50809-kube-api-access-7kwwj\" (UniqueName: \"kubernetes.io/projected/293b1916-b37f-46c7-8ce6-b9efa6a50809-kube-api-access-7kwwj\") pod \"293b1916-b37f-46c7-8ce6-b9efa6a50809\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " Oct 30 19:10:59.863879 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.863850 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-podinfo\") pod \"293b1916-b37f-46c7-8ce6-b9efa6a50809\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " Oct 30 19:10:59.864014 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.863994 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-workload-socket\") pod \"293b1916-b37f-46c7-8ce6-b9efa6a50809\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " Oct 30 19:10:59.864104 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.864055 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-workload-certs\") pod \"293b1916-b37f-46c7-8ce6-b9efa6a50809\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " Oct 30 19:10:59.864163 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.864111 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-credential-socket\") pod \"293b1916-b37f-46c7-8ce6-b9efa6a50809\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " Oct 30 19:10:59.864220 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.864171 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-token\" (UniqueName: \"kubernetes.io/projected/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-token\") pod \"293b1916-b37f-46c7-8ce6-b9efa6a50809\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " Oct 30 19:10:59.864274 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.864261 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/293b1916-b37f-46c7-8ce6-b9efa6a50809-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/293b1916-b37f-46c7-8ce6-b9efa6a50809-istiod-ca-cert\") pod \"293b1916-b37f-46c7-8ce6-b9efa6a50809\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " Oct 30 19:10:59.864328 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.864295 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-data\") pod \"293b1916-b37f-46c7-8ce6-b9efa6a50809\" (UID: \"293b1916-b37f-46c7-8ce6-b9efa6a50809\") " Oct 30 19:10:59.865221 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.865062 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-workload-certs" pod "293b1916-b37f-46c7-8ce6-b9efa6a50809" (UID: "293b1916-b37f-46c7-8ce6-b9efa6a50809"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:10:59.865221 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.865171 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-workload-socket" pod "293b1916-b37f-46c7-8ce6-b9efa6a50809" (UID: "293b1916-b37f-46c7-8ce6-b9efa6a50809"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:10:59.865458 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.865406 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/293b1916-b37f-46c7-8ce6-b9efa6a50809-istiod-ca-cert" pod "293b1916-b37f-46c7-8ce6-b9efa6a50809" (UID: "293b1916-b37f-46c7-8ce6-b9efa6a50809"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:10:59.865620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.865593 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-data" pod "293b1916-b37f-46c7-8ce6-b9efa6a50809" (UID: "293b1916-b37f-46c7-8ce6-b9efa6a50809"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:10:59.865758 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.865724 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-credential-socket" pod "293b1916-b37f-46c7-8ce6-b9efa6a50809" (UID: "293b1916-b37f-46c7-8ce6-b9efa6a50809"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:10:59.866327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.866291 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-envoy" pod "293b1916-b37f-46c7-8ce6-b9efa6a50809" (UID: "293b1916-b37f-46c7-8ce6-b9efa6a50809"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:10:59.866541 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.866517 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/293b1916-b37f-46c7-8ce6-b9efa6a50809-kube-api-access-7kwwj" pod "293b1916-b37f-46c7-8ce6-b9efa6a50809" (UID: "293b1916-b37f-46c7-8ce6-b9efa6a50809"). InnerVolumeSpecName "kube-api-access-7kwwj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:10:59.866651 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.866636 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-podinfo" pod "293b1916-b37f-46c7-8ce6-b9efa6a50809" (UID: "293b1916-b37f-46c7-8ce6-b9efa6a50809"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:10:59.867689 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.867668 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-token" pod "293b1916-b37f-46c7-8ce6-b9efa6a50809" (UID: "293b1916-b37f-46c7-8ce6-b9efa6a50809"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:10:59.974901 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.974749 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kwwj\" (UniqueName: \"kubernetes.io/projected/293b1916-b37f-46c7-8ce6-b9efa6a50809-kube-api-access-7kwwj\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:10:59.974901 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.974782 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:10:59.974901 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.974798 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:10:59.974901 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.974813 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:10:59.974901 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.974827 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:10:59.974901 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.974841 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:10:59.974901 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.974854 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/293b1916-b37f-46c7-8ce6-b9efa6a50809-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:10:59.974901 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.974867 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:10:59.974901 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.974880 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/293b1916-b37f-46c7-8ce6-b9efa6a50809-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:10:59.979918 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.979876 5894 generic.go:356] "Generic (PLEG): container finished" podID="293b1916-b37f-46c7-8ce6-b9efa6a50809" containerID="0c03b664de0e6d627b50ffbf3913a252a173cf13b971f1b49bd26476f1c1bc09" exitCode=0 Oct 30 19:10:59.980140 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.979943 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" Oct 30 19:10:59.980140 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.979993 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" event={"ID":"293b1916-b37f-46c7-8ce6-b9efa6a50809","Type":"ContainerDied","Data":"0c03b664de0e6d627b50ffbf3913a252a173cf13b971f1b49bd26476f1c1bc09"} Oct 30 19:10:59.980283 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.980150 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df" event={"ID":"293b1916-b37f-46c7-8ce6-b9efa6a50809","Type":"ContainerDied","Data":"3ae39c9d7678a99d8d7e7eb04611baa548f87a234913f4f9651de3784e6f81c2"} Oct 30 19:10:59.980283 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:10:59.980187 5894 scope.go:122] "RemoveContainer" containerID="0c03b664de0e6d627b50ffbf3913a252a173cf13b971f1b49bd26476f1c1bc09" Oct 30 19:11:00.039511 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.039433 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:00.064055 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.064034 5894 scope.go:122] "RemoveContainer" containerID="0c03b664de0e6d627b50ffbf3913a252a173cf13b971f1b49bd26476f1c1bc09" Oct 30 19:11:00.064409 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:11:00.064383 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0c03b664de0e6d627b50ffbf3913a252a173cf13b971f1b49bd26476f1c1bc09\": container with ID starting with 0c03b664de0e6d627b50ffbf3913a252a173cf13b971f1b49bd26476f1c1bc09 not found: ID does not exist" containerID="0c03b664de0e6d627b50ffbf3913a252a173cf13b971f1b49bd26476f1c1bc09" Oct 30 19:11:00.064470 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.064421 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0c03b664de0e6d627b50ffbf3913a252a173cf13b971f1b49bd26476f1c1bc09"} err="failed to get container status \"0c03b664de0e6d627b50ffbf3913a252a173cf13b971f1b49bd26476f1c1bc09\": rpc error: code = NotFound desc = could not find container \"0c03b664de0e6d627b50ffbf3913a252a173cf13b971f1b49bd26476f1c1bc09\": container with ID starting with 0c03b664de0e6d627b50ffbf3913a252a173cf13b971f1b49bd26476f1c1bc09 not found: ID does not exist" Oct 30 19:11:00.155807 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.155736 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df"] Oct 30 19:11:00.160240 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.160209 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-5b9df"] Oct 30 19:11:00.239603 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.235471 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:00.362777 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.362749 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 19:11:00.422829 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.422794 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:00.485348 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.485316 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-token\" (UniqueName: \"kubernetes.io/projected/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-token\") pod \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " Oct 30 19:11:00.485596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.485562 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-podinfo\") pod \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " Oct 30 19:11:00.485781 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.485766 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-workload-socket\") pod \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " Oct 30 19:11:00.485912 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.485897 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-kube-api-access-vqqg7\" (UniqueName: \"kubernetes.io/projected/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-kube-api-access-vqqg7\") pod \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " Oct 30 19:11:00.486146 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.486116 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-workload-certs\") pod \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " Oct 30 19:11:00.486286 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.486155 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-envoy\") pod \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " Oct 30 19:11:00.486286 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.486212 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-credential-socket\") pod \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " Oct 30 19:11:00.486286 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.486277 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-data\") pod \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " Oct 30 19:11:00.486461 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.486354 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istiod-ca-cert\") pod \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\" (UID: \"0133d032-d0fb-4623-8c0e-4ddc8b765c6d\") " Oct 30 19:11:00.486953 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.486892 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-workload-certs" pod "0133d032-d0fb-4623-8c0e-4ddc8b765c6d" (UID: "0133d032-d0fb-4623-8c0e-4ddc8b765c6d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:11:00.487454 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.487418 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istiod-ca-cert" pod "0133d032-d0fb-4623-8c0e-4ddc8b765c6d" (UID: "0133d032-d0fb-4623-8c0e-4ddc8b765c6d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:11:00.487564 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.486989 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-data" pod "0133d032-d0fb-4623-8c0e-4ddc8b765c6d" (UID: "0133d032-d0fb-4623-8c0e-4ddc8b765c6d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:11:00.487783 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.487760 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-workload-socket" pod "0133d032-d0fb-4623-8c0e-4ddc8b765c6d" (UID: "0133d032-d0fb-4623-8c0e-4ddc8b765c6d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:11:00.488935 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.488866 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-credential-socket" pod "0133d032-d0fb-4623-8c0e-4ddc8b765c6d" (UID: "0133d032-d0fb-4623-8c0e-4ddc8b765c6d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:11:00.490304 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.490279 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-envoy" pod "0133d032-d0fb-4623-8c0e-4ddc8b765c6d" (UID: "0133d032-d0fb-4623-8c0e-4ddc8b765c6d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:11:00.490537 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.490504 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-podinfo" pod "0133d032-d0fb-4623-8c0e-4ddc8b765c6d" (UID: "0133d032-d0fb-4623-8c0e-4ddc8b765c6d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:11:00.491797 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.491768 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-kube-api-access-vqqg7" pod "0133d032-d0fb-4623-8c0e-4ddc8b765c6d" (UID: "0133d032-d0fb-4623-8c0e-4ddc8b765c6d"). InnerVolumeSpecName "kube-api-access-vqqg7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:11:00.493100 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.493072 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-token" pod "0133d032-d0fb-4623-8c0e-4ddc8b765c6d" (UID: "0133d032-d0fb-4623-8c0e-4ddc8b765c6d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:11:00.588544 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.588505 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:11:00.588544 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.588535 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:11:00.588787 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.588549 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:11:00.588787 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.588563 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:11:00.588787 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.588615 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vqqg7\" (UniqueName: \"kubernetes.io/projected/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-kube-api-access-vqqg7\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:11:00.588787 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.588628 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:11:00.588787 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.588640 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:11:00.588787 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.588653 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:11:00.588787 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.588665 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0133d032-d0fb-4623-8c0e-4ddc8b765c6d-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:11:00.988776 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.988741 5894 generic.go:356] "Generic (PLEG): container finished" podID="0133d032-d0fb-4623-8c0e-4ddc8b765c6d" containerID="5941388e9c1d6ca368b3b98ca036b6bdf8980a374e924171797f95fe1850d341" exitCode=0 Oct 30 19:11:00.989262 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.988795 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" Oct 30 19:11:00.989262 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.988819 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" event={"ID":"0133d032-d0fb-4623-8c0e-4ddc8b765c6d","Type":"ContainerDied","Data":"5941388e9c1d6ca368b3b98ca036b6bdf8980a374e924171797f95fe1850d341"} Oct 30 19:11:00.989262 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.988869 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr" event={"ID":"0133d032-d0fb-4623-8c0e-4ddc8b765c6d","Type":"ContainerDied","Data":"9761171fbd0846fc8fd5017723935b2f7b7c8cc9cfa53e4b2ff4d3f0dc075b8d"} Oct 30 19:11:00.989262 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.988891 5894 scope.go:122] "RemoveContainer" containerID="5941388e9c1d6ca368b3b98ca036b6bdf8980a374e924171797f95fe1850d341" Oct 30 19:11:00.999037 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.999021 5894 scope.go:122] "RemoveContainer" containerID="5941388e9c1d6ca368b3b98ca036b6bdf8980a374e924171797f95fe1850d341" Oct 30 19:11:00.999295 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:11:00.999278 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5941388e9c1d6ca368b3b98ca036b6bdf8980a374e924171797f95fe1850d341\": container with ID starting with 5941388e9c1d6ca368b3b98ca036b6bdf8980a374e924171797f95fe1850d341 not found: ID does not exist" containerID="5941388e9c1d6ca368b3b98ca036b6bdf8980a374e924171797f95fe1850d341" Oct 30 19:11:00.999343 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:00.999302 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5941388e9c1d6ca368b3b98ca036b6bdf8980a374e924171797f95fe1850d341"} err="failed to get container status \"5941388e9c1d6ca368b3b98ca036b6bdf8980a374e924171797f95fe1850d341\": rpc error: code = NotFound desc = could not find container \"5941388e9c1d6ca368b3b98ca036b6bdf8980a374e924171797f95fe1850d341\": container with ID starting with 5941388e9c1d6ca368b3b98ca036b6bdf8980a374e924171797f95fe1850d341 not found: ID does not exist" Oct 30 19:11:01.154438 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:01.154394 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr"] Oct 30 19:11:01.204178 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:01.204145 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-v7dhr"] Oct 30 19:11:01.652049 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:01.652009 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0133d032-d0fb-4623-8c0e-4ddc8b765c6d" path="/var/lib/kubelet/pods/0133d032-d0fb-4623-8c0e-4ddc8b765c6d/volumes" Oct 30 19:11:01.652569 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:01.652554 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="293b1916-b37f-46c7-8ce6-b9efa6a50809" path="/var/lib/kubelet/pods/293b1916-b37f-46c7-8ce6-b9efa6a50809/volumes" Oct 30 19:11:05.067296 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:05.067209 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:10.067745 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:10.067657 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:14.181841 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:14.181807 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:14.896612 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:14.896556 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:18.987373 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:18.987338 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-9104-19be74"] Oct 30 19:11:18.988004 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:18.987989 5894 cpu_manager.go:414] "containerMap: removing container" podUID="ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" containerName="istio-proxy" Oct 30 19:11:18.988069 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:18.988005 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" containerName="istio-proxy" Oct 30 19:11:18.988069 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:18.988024 5894 cpu_manager.go:414] "containerMap: removing container" podUID="0133d032-d0fb-4623-8c0e-4ddc8b765c6d" containerName="istio-proxy" Oct 30 19:11:18.988069 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:18.988029 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0133d032-d0fb-4623-8c0e-4ddc8b765c6d" containerName="istio-proxy" Oct 30 19:11:18.988069 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:18.988057 5894 cpu_manager.go:414] "containerMap: removing container" podUID="ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" containerName="istio-proxy" Oct 30 19:11:18.988069 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:18.988063 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" containerName="istio-proxy" Oct 30 19:11:18.988232 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:18.988080 5894 cpu_manager.go:414] "containerMap: removing container" podUID="293b1916-b37f-46c7-8ce6-b9efa6a50809" containerName="istio-proxy" Oct 30 19:11:18.988232 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:18.988085 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="293b1916-b37f-46c7-8ce6-b9efa6a50809" containerName="istio-proxy" Oct 30 19:11:18.988232 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:18.988176 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="0133d032-d0fb-4623-8c0e-4ddc8b765c6d" containerName="istio-proxy" Oct 30 19:11:18.988232 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:18.988189 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" containerName="istio-proxy" Oct 30 19:11:18.988232 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:18.988197 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="293b1916-b37f-46c7-8ce6-b9efa6a50809" containerName="istio-proxy" Oct 30 19:11:18.988232 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:18.988207 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="ef4b2f6f-0e65-4dd4-b028-c50314c9e7e4" containerName="istio-proxy" Oct 30 19:11:18.991725 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:18.991707 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9104-19be74" Oct 30 19:11:19.046154 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:19.046120 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9104-19be74"] Oct 30 19:11:19.091964 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:19.091932 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-7v9w4\"" Oct 30 19:11:19.109774 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:19.109738 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5p946\" (UniqueName: \"kubernetes.io/projected/7e428556-02f4-4970-953d-90abac4d456d-kube-api-access-5p946\") pod \"test-curl-9104-19be74\" (UID: \"7e428556-02f4-4970-953d-90abac4d456d\") " pod="opendatahub/test-curl-9104-19be74" Oct 30 19:11:19.211104 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:19.211075 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5p946\" (UniqueName: \"kubernetes.io/projected/7e428556-02f4-4970-953d-90abac4d456d-kube-api-access-5p946\") pod \"test-curl-9104-19be74\" (UID: \"7e428556-02f4-4970-953d-90abac4d456d\") " pod="opendatahub/test-curl-9104-19be74" Oct 30 19:11:19.267692 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:19.267627 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5p946\" (UniqueName: \"kubernetes.io/projected/7e428556-02f4-4970-953d-90abac4d456d-kube-api-access-5p946\") pod \"test-curl-9104-19be74\" (UID: \"7e428556-02f4-4970-953d-90abac4d456d\") " pod="opendatahub/test-curl-9104-19be74" Oct 30 19:11:19.304699 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:19.304663 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9104-19be74" Oct 30 19:11:19.548613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:19.548500 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:19.677524 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:19.677498 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-9104-19be74"] Oct 30 19:11:19.679510 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:11:19.679481 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7e428556_02f4_4970_953d_90abac4d456d.slice/crio-a01a6f967ea4a235b6240136014649c9bed9f6fa0baa8cd32757cc06b204bd42 WatchSource:0}: Error finding container a01a6f967ea4a235b6240136014649c9bed9f6fa0baa8cd32757cc06b204bd42: Status 404 returned error can't find the container with id a01a6f967ea4a235b6240136014649c9bed9f6fa0baa8cd32757cc06b204bd42 Oct 30 19:11:20.011939 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:20.011902 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:20.119000 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:11:20.118909 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:11:20.144150 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:20.144107 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9104-19be74" event={"ID":"7e428556-02f4-4970-953d-90abac4d456d","Type":"ContainerStarted","Data":"e588708391f26915fdc550524639b8402e448e8c9d754751bb0aed84d86030a1"} Oct 30 19:11:20.144150 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:20.144153 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-9104-19be74" event={"ID":"7e428556-02f4-4970-953d-90abac4d456d","Type":"ContainerStarted","Data":"a01a6f967ea4a235b6240136014649c9bed9f6fa0baa8cd32757cc06b204bd42"} Oct 30 19:11:20.255118 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:20.255064 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/test-curl-9104-19be74" podStartSLOduration=2.255049517 podStartE2EDuration="2.255049517s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:11:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:11:20.254640997 +0000 UTC m=+1950.744768823" watchObservedRunningTime="2026-10-30 19:11:20.255049517 +0000 UTC m=+1950.745177336" Oct 30 19:11:21.665755 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:21.665722 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-9104-19be74"] Oct 30 19:11:21.670970 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:21.670933 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-9104-19be74"] Oct 30 19:11:22.157484 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:22.157427 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/test-curl-9104-19be74" podUID="7e428556-02f4-4970-953d-90abac4d456d" containerName="test-curl-9104-19be74" containerID="cri-o://e588708391f26915fdc550524639b8402e448e8c9d754751bb0aed84d86030a1" gracePeriod=2 Oct 30 19:11:22.208933 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:11:22.208886 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9104-19be74\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="7e428556-02f4-4970-953d-90abac4d456d" pod="opendatahub/test-curl-9104-19be74" Oct 30 19:11:22.984238 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:22.984206 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:24.340821 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:24.340784 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:24.483282 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:24.483255 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9104-19be74" Oct 30 19:11:24.601197 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:24.601163 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7e428556-02f4-4970-953d-90abac4d456d-kube-api-access-5p946\" (UniqueName: \"kubernetes.io/projected/7e428556-02f4-4970-953d-90abac4d456d-kube-api-access-5p946\") pod \"7e428556-02f4-4970-953d-90abac4d456d\" (UID: \"7e428556-02f4-4970-953d-90abac4d456d\") " Oct 30 19:11:24.603337 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:24.603310 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e428556-02f4-4970-953d-90abac4d456d-kube-api-access-5p946" pod "7e428556-02f4-4970-953d-90abac4d456d" (UID: "7e428556-02f4-4970-953d-90abac4d456d"). InnerVolumeSpecName "kube-api-access-5p946". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:11:24.703310 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:24.703283 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5p946\" (UniqueName: \"kubernetes.io/projected/7e428556-02f4-4970-953d-90abac4d456d-kube-api-access-5p946\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:11:25.178463 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:25.178426 5894 generic.go:356] "Generic (PLEG): container finished" podID="7e428556-02f4-4970-953d-90abac4d456d" containerID="e588708391f26915fdc550524639b8402e448e8c9d754751bb0aed84d86030a1" exitCode=137 Oct 30 19:11:25.178668 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:25.178469 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-9104-19be74" Oct 30 19:11:25.178668 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:25.178519 5894 scope.go:122] "RemoveContainer" containerID="e588708391f26915fdc550524639b8402e448e8c9d754751bb0aed84d86030a1" Oct 30 19:11:25.188769 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:25.188751 5894 scope.go:122] "RemoveContainer" containerID="e588708391f26915fdc550524639b8402e448e8c9d754751bb0aed84d86030a1" Oct 30 19:11:25.189092 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:11:25.189072 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e588708391f26915fdc550524639b8402e448e8c9d754751bb0aed84d86030a1\": container with ID starting with e588708391f26915fdc550524639b8402e448e8c9d754751bb0aed84d86030a1 not found: ID does not exist" containerID="e588708391f26915fdc550524639b8402e448e8c9d754751bb0aed84d86030a1" Oct 30 19:11:25.189156 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:25.189102 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e588708391f26915fdc550524639b8402e448e8c9d754751bb0aed84d86030a1"} err="failed to get container status \"e588708391f26915fdc550524639b8402e448e8c9d754751bb0aed84d86030a1\": rpc error: code = NotFound desc = could not find container \"e588708391f26915fdc550524639b8402e448e8c9d754751bb0aed84d86030a1\": container with ID starting with e588708391f26915fdc550524639b8402e448e8c9d754751bb0aed84d86030a1 not found: ID does not exist" Oct 30 19:11:25.239299 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:11:25.239272 5894 status_manager.go:1045] "Failed to get status for pod" err="pods \"test-curl-9104-19be74\" is forbidden: User \"system:node:ip-10-0-132-122.us-west-1.compute.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-122.us-west-1.compute.internal' and this object" podUID="7e428556-02f4-4970-953d-90abac4d456d" pod="opendatahub/test-curl-9104-19be74" Oct 30 19:11:25.651413 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:25.651384 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7e428556-02f4-4970-953d-90abac4d456d" path="/var/lib/kubelet/pods/7e428556-02f4-4970-953d-90abac4d456d/volumes" Oct 30 19:11:29.476635 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:29.472086 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:34.875597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:34.874260 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:35.493667 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:35.479892 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:36.244238 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:36.244203 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:36.925598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:36.908938 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:37.508986 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:37.508952 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:39.057688 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:39.057658 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:39.269960 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:39.269911 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:40.390601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:40.389992 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:42.061883 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:42.061843 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:42.260525 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:42.260497 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:45.260314 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:45.260273 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:45.454516 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:45.454483 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:46.014081 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:46.014039 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:50.119002 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:11:50.118958 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:11:51.202349 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:51.201671 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:54.085547 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:54.085513 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t"] Oct 30 19:11:54.085983 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:54.085746 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" podUID="0464dd8d-01e7-4ef3-a866-d77fbe31d99a" containerName="istio-proxy" containerID="cri-o://94c54965edc3da8607376c1f49837a3b4799f6b829daef64632d841ea173af92" gracePeriod=30 Oct 30 19:11:56.082762 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:56.082719 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:56.262341 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:56.262310 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:56.433738 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:56.433707 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:58.139396 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:58.139354 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:58.373975 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:58.348737 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:11:59.522170 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.521967 5894 generic.go:356] "Generic (PLEG): container finished" podID="0464dd8d-01e7-4ef3-a866-d77fbe31d99a" containerID="94c54965edc3da8607376c1f49837a3b4799f6b829daef64632d841ea173af92" exitCode=0 Oct 30 19:11:59.522170 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.522092 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" event={"ID":"0464dd8d-01e7-4ef3-a866-d77fbe31d99a","Type":"ContainerDied","Data":"94c54965edc3da8607376c1f49837a3b4799f6b829daef64632d841ea173af92"} Oct 30 19:11:59.807839 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.807808 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 19:11:59.896408 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.896375 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-podinfo\") pod \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " Oct 30 19:11:59.896613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.896439 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-envoy\") pod \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " Oct 30 19:11:59.896613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.896478 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-data\") pod \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " Oct 30 19:11:59.896613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.896549 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istiod-ca-cert\") pod \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " Oct 30 19:11:59.896802 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.896780 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-token\" (UniqueName: \"kubernetes.io/projected/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-token\") pod \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " Oct 30 19:11:59.896867 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.896816 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-credential-socket\") pod \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " Oct 30 19:11:59.896867 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.896841 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-data" pod "0464dd8d-01e7-4ef3-a866-d77fbe31d99a" (UID: "0464dd8d-01e7-4ef3-a866-d77fbe31d99a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:11:59.896986 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.896882 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-workload-certs\") pod \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " Oct 30 19:11:59.896986 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.896947 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istiod-ca-cert" pod "0464dd8d-01e7-4ef3-a866-d77fbe31d99a" (UID: "0464dd8d-01e7-4ef3-a866-d77fbe31d99a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:11:59.896986 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.896950 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-kube-api-access-rjhvz\" (UniqueName: \"kubernetes.io/projected/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-kube-api-access-rjhvz\") pod \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " Oct 30 19:11:59.897166 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.897142 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-workload-socket\") pod \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\" (UID: \"0464dd8d-01e7-4ef3-a866-d77fbe31d99a\") " Oct 30 19:11:59.897234 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.897143 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-credential-socket" pod "0464dd8d-01e7-4ef3-a866-d77fbe31d99a" (UID: "0464dd8d-01e7-4ef3-a866-d77fbe31d99a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:11:59.897234 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.897205 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-workload-certs" pod "0464dd8d-01e7-4ef3-a866-d77fbe31d99a" (UID: "0464dd8d-01e7-4ef3-a866-d77fbe31d99a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:11:59.897366 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.897346 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-workload-socket" pod "0464dd8d-01e7-4ef3-a866-d77fbe31d99a" (UID: "0464dd8d-01e7-4ef3-a866-d77fbe31d99a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:11:59.897933 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.897907 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:11:59.898071 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.897938 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:11:59.898071 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.897954 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:11:59.898071 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.897972 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:11:59.898071 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.897987 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:11:59.898824 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.898789 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-podinfo" pod "0464dd8d-01e7-4ef3-a866-d77fbe31d99a" (UID: "0464dd8d-01e7-4ef3-a866-d77fbe31d99a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:11:59.899011 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.898990 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-token" pod "0464dd8d-01e7-4ef3-a866-d77fbe31d99a" (UID: "0464dd8d-01e7-4ef3-a866-d77fbe31d99a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:11:59.899071 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.898997 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-envoy" pod "0464dd8d-01e7-4ef3-a866-d77fbe31d99a" (UID: "0464dd8d-01e7-4ef3-a866-d77fbe31d99a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:11:59.899217 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:11:59.899200 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-kube-api-access-rjhvz" pod "0464dd8d-01e7-4ef3-a866-d77fbe31d99a" (UID: "0464dd8d-01e7-4ef3-a866-d77fbe31d99a"). InnerVolumeSpecName "kube-api-access-rjhvz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:12:00.006609 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:00.000604 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rjhvz\" (UniqueName: \"kubernetes.io/projected/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-kube-api-access-rjhvz\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:12:00.006609 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:00.000658 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:12:00.006609 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:00.000677 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:12:00.006609 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:00.000691 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0464dd8d-01e7-4ef3-a866-d77fbe31d99a-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:12:00.530007 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:00.529975 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" Oct 30 19:12:00.530007 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:00.529984 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t" event={"ID":"0464dd8d-01e7-4ef3-a866-d77fbe31d99a","Type":"ContainerDied","Data":"32ad04e5b21d03b7f3dcefab7463b67c67409a95ceff489b24e9bbff282dcf4a"} Oct 30 19:12:00.530478 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:00.530069 5894 scope.go:122] "RemoveContainer" containerID="94c54965edc3da8607376c1f49837a3b4799f6b829daef64632d841ea173af92" Oct 30 19:12:00.750900 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:00.750867 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t"] Oct 30 19:12:00.754047 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:00.754020 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-kpj9t"] Oct 30 19:12:01.606658 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:01.602998 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:01.687769 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:01.687733 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0464dd8d-01e7-4ef3-a866-d77fbe31d99a" path="/var/lib/kubelet/pods/0464dd8d-01e7-4ef3-a866-d77fbe31d99a/volumes" Oct 30 19:12:06.314480 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:06.314437 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:06.508948 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:06.508915 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:06.697997 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:06.697959 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:09.828861 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:09.828829 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:10.071381 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:10.071353 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:11.423859 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:11.423821 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:16.200378 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:16.200338 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:20.119386 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:12:20.119343 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:12:21.029063 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:21.028930 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:21.571811 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:21.571775 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:23.298819 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:23.298766 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:25.699950 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:25.699902 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:30.451154 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:30.451115 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:35.524546 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:35.524505 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:40.275733 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:40.274793 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:44.612350 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:44.612315 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:45.075516 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:45.075484 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:45.541196 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:45.541153 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:46.032597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:46.032019 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:46.461333 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:46.461293 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:46.669011 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:46.667309 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:50.071234 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:50.071175 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:50.175304 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:12:50.175268 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:12:50.364634 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:50.364548 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:50.554291 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:50.554257 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:53.361316 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:53.361286 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:53.629087 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:53.628997 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:54.105905 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:54.105867 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp"] Oct 30 19:12:54.107262 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:54.107217 5894 cpu_manager.go:414] "containerMap: removing container" podUID="0464dd8d-01e7-4ef3-a866-d77fbe31d99a" containerName="istio-proxy" Oct 30 19:12:54.107437 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:54.107421 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0464dd8d-01e7-4ef3-a866-d77fbe31d99a" containerName="istio-proxy" Oct 30 19:12:54.107552 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:54.107540 5894 cpu_manager.go:414] "containerMap: removing container" podUID="7e428556-02f4-4970-953d-90abac4d456d" containerName="test-curl-9104-19be74" Oct 30 19:12:54.107677 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:54.107663 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7e428556-02f4-4970-953d-90abac4d456d" containerName="test-curl-9104-19be74" Oct 30 19:12:54.108057 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:54.108040 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="0464dd8d-01e7-4ef3-a866-d77fbe31d99a" containerName="istio-proxy" Oct 30 19:12:54.108173 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:54.108160 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="7e428556-02f4-4970-953d-90abac4d456d" containerName="test-curl-9104-19be74" Oct 30 19:12:54.150215 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:54.150183 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp"] Oct 30 19:12:54.150485 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:54.150467 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp" Oct 30 19:12:54.202636 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:54.202607 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-8da6e2-dockercfg-fsjf9\"" Oct 30 19:12:54.213622 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:54.213558 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d683e67-87f6-4d4b-a208-0ebea816c5a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp\" (UID: \"9d683e67-87f6-4d4b-a208-0ebea816c5a7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp" Oct 30 19:12:54.213885 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:54.213869 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sl2fs\" (UniqueName: \"kubernetes.io/projected/9d683e67-87f6-4d4b-a208-0ebea816c5a7-kube-api-access-sl2fs\") pod \"payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp\" (UID: \"9d683e67-87f6-4d4b-a208-0ebea816c5a7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp" Oct 30 19:12:54.252309 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:54.252283 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-8da6e2\"" Oct 30 19:12:54.314915 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:54.314876 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sl2fs\" (UniqueName: \"kubernetes.io/projected/9d683e67-87f6-4d4b-a208-0ebea816c5a7-kube-api-access-sl2fs\") pod \"payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp\" (UID: \"9d683e67-87f6-4d4b-a208-0ebea816c5a7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp" Oct 30 19:12:54.315092 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:54.314931 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d683e67-87f6-4d4b-a208-0ebea816c5a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp\" (UID: \"9d683e67-87f6-4d4b-a208-0ebea816c5a7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp" Oct 30 19:12:54.315652 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:54.315634 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d683e67-87f6-4d4b-a208-0ebea816c5a7-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp\" (UID: \"9d683e67-87f6-4d4b-a208-0ebea816c5a7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp" Oct 30 19:12:54.371533 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:54.371467 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sl2fs\" (UniqueName: \"kubernetes.io/projected/9d683e67-87f6-4d4b-a208-0ebea816c5a7-kube-api-access-sl2fs\") pod \"payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp\" (UID: \"9d683e67-87f6-4d4b-a208-0ebea816c5a7\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp" Oct 30 19:12:54.473801 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:54.473771 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp" Oct 30 19:12:54.856949 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:54.856917 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp"] Oct 30 19:12:54.857836 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:12:54.857805 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d683e67_87f6_4d4b_a208_0ebea816c5a7.slice/crio-1bf8ca764547ae5e900f820dd5690b544ae62675536b6716145eafefba27a382 WatchSource:0}: Error finding container 1bf8ca764547ae5e900f820dd5690b544ae62675536b6716145eafefba27a382: Status 404 returned error can't find the container with id 1bf8ca764547ae5e900f820dd5690b544ae62675536b6716145eafefba27a382 Oct 30 19:12:55.152724 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:55.152686 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:12:55.200268 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:55.200230 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp" event={"ID":"9d683e67-87f6-4d4b-a208-0ebea816c5a7","Type":"ContainerStarted","Data":"fcc84f87901701cca3cff82cb7b27fa1229e305713851f9d1e94bd8346985db3"} Oct 30 19:12:55.200463 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:55.200307 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp" event={"ID":"9d683e67-87f6-4d4b-a208-0ebea816c5a7","Type":"ContainerStarted","Data":"1bf8ca764547ae5e900f820dd5690b544ae62675536b6716145eafefba27a382"} Oct 30 19:12:55.200682 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:55.200660 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp" Oct 30 19:12:55.316224 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:55.316171 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp" podStartSLOduration=1.316158168 podStartE2EDuration="1.316158168s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:12:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:12:55.315030779 +0000 UTC m=+2045.805158619" watchObservedRunningTime="2026-10-30 19:12:55.316158168 +0000 UTC m=+2045.806285988" Oct 30 19:12:59.930596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:12:59.903443 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:01.059053 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:01.059020 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:04.681599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:04.675295 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:06.210925 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:06.210895 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp" Oct 30 19:13:09.202035 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:09.202000 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:09.375773 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:09.375736 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:09.696910 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:09.696863 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:10.060289 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:10.060203 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69"] Oct 30 19:13:10.060564 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:10.060501 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" podUID="886729d6-2218-45ce-b082-64fe59f772a2" containerName="istio-proxy" containerID="cri-o://8b052e8b22a76b049786fe3e5a915cf20450d5ed3552371e8f4f82700e535bf9" gracePeriod=30 Oct 30 19:13:10.111378 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:10.111343 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg"] Oct 30 19:13:10.111641 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:10.111612 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" podUID="5a3f5720-0f19-4f6d-9609-1924e6d45b3e" containerName="istio-proxy" containerID="cri-o://fd5be8b365afbc1c2379b5192a44abe351aeae024d322abd470817ae5ba61023" gracePeriod=30 Oct 30 19:13:10.768613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:10.768568 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq"] Oct 30 19:13:10.769031 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:10.768799 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" podUID="43b1379e-42d3-4913-ab29-d766689a4a90" containerName="istio-proxy" containerID="cri-o://469a6bab9d326996c01f9c9d782beebc8e837f38ff66edadf4f948d0931b05ba" gracePeriod=30 Oct 30 19:13:10.772382 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:10.771822 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs"] Oct 30 19:13:10.772382 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:10.772035 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" podUID="11f90211-60b6-41d7-a44f-45198e9eb898" containerName="istio-proxy" containerID="cri-o://a3021eab31d5bdaff70b125da4f2f54a46fe04d5694027883eb2ca42e689b4ac" gracePeriod=30 Oct 30 19:13:10.870443 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:10.870394 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh"] Oct 30 19:13:10.870738 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:10.870694 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" podUID="48ae9925-3d54-48a5-af7a-d6bc186ea365" containerName="istio-proxy" containerID="cri-o://b0a98eee3b434de1693b0eee3a355b5bad53d4cddcb4268e8f5687abff3fc7b7" gracePeriod=30 Oct 30 19:13:12.764956 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:12.764923 5894 patch_prober.go:29] interesting pod/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:13:12.765325 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:12.764995 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" podUID="43b1379e-42d3-4913-ab29-d766689a4a90" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:13:13.640675 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:13.640638 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:13.855780 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:13.855740 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:13:13.856381 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:13.856342 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" podUID="5a3f5720-0f19-4f6d-9609-1924e6d45b3e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:13:14.815253 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:14.815216 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:15.044830 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.044789 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:13:15.045281 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.044857 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" podUID="48ae9925-3d54-48a5-af7a-d6bc186ea365" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:13:15.289363 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.289219 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:15.391379 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.389212 5894 generic.go:356] "Generic (PLEG): container finished" podID="5a3f5720-0f19-4f6d-9609-1924e6d45b3e" containerID="fd5be8b365afbc1c2379b5192a44abe351aeae024d322abd470817ae5ba61023" exitCode=0 Oct 30 19:13:15.391379 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.389296 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" event={"ID":"5a3f5720-0f19-4f6d-9609-1924e6d45b3e","Type":"ContainerDied","Data":"fd5be8b365afbc1c2379b5192a44abe351aeae024d322abd470817ae5ba61023"} Oct 30 19:13:15.391379 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.391126 5894 generic.go:356] "Generic (PLEG): container finished" podID="886729d6-2218-45ce-b082-64fe59f772a2" containerID="8b052e8b22a76b049786fe3e5a915cf20450d5ed3552371e8f4f82700e535bf9" exitCode=0 Oct 30 19:13:15.391379 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.391285 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" event={"ID":"886729d6-2218-45ce-b082-64fe59f772a2","Type":"ContainerDied","Data":"8b052e8b22a76b049786fe3e5a915cf20450d5ed3552371e8f4f82700e535bf9"} Oct 30 19:13:15.562648 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.562617 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:13:15.572714 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.572682 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:15.599528 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.599501 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-istio-envoy\") pod \"886729d6-2218-45ce-b082-64fe59f772a2\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " Oct 30 19:13:15.599661 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.599546 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/886729d6-2218-45ce-b082-64fe59f772a2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/886729d6-2218-45ce-b082-64fe59f772a2-istio-podinfo\") pod \"886729d6-2218-45ce-b082-64fe59f772a2\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " Oct 30 19:13:15.599790 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.599772 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-istio-data\") pod \"886729d6-2218-45ce-b082-64fe59f772a2\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " Oct 30 19:13:15.599862 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.599843 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-workload-socket\") pod \"886729d6-2218-45ce-b082-64fe59f772a2\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " Oct 30 19:13:15.599922 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.599900 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/886729d6-2218-45ce-b082-64fe59f772a2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/886729d6-2218-45ce-b082-64fe59f772a2-istiod-ca-cert\") pod \"886729d6-2218-45ce-b082-64fe59f772a2\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " Oct 30 19:13:15.599997 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.599979 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-credential-socket\") pod \"886729d6-2218-45ce-b082-64fe59f772a2\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " Oct 30 19:13:15.600056 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.600008 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-istio-data" pod "886729d6-2218-45ce-b082-64fe59f772a2" (UID: "886729d6-2218-45ce-b082-64fe59f772a2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:15.600056 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.600019 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/886729d6-2218-45ce-b082-64fe59f772a2-istio-token\" (UniqueName: \"kubernetes.io/projected/886729d6-2218-45ce-b082-64fe59f772a2-istio-token\") pod \"886729d6-2218-45ce-b082-64fe59f772a2\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " Oct 30 19:13:15.600172 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.600101 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/886729d6-2218-45ce-b082-64fe59f772a2-kube-api-access-sps6v\" (UniqueName: \"kubernetes.io/projected/886729d6-2218-45ce-b082-64fe59f772a2-kube-api-access-sps6v\") pod \"886729d6-2218-45ce-b082-64fe59f772a2\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " Oct 30 19:13:15.600172 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.600113 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-workload-socket" pod "886729d6-2218-45ce-b082-64fe59f772a2" (UID: "886729d6-2218-45ce-b082-64fe59f772a2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:15.600277 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.600242 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-credential-socket" pod "886729d6-2218-45ce-b082-64fe59f772a2" (UID: "886729d6-2218-45ce-b082-64fe59f772a2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:15.600336 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.600314 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-workload-certs\") pod \"886729d6-2218-45ce-b082-64fe59f772a2\" (UID: \"886729d6-2218-45ce-b082-64fe59f772a2\") " Oct 30 19:13:15.600402 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.600353 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/886729d6-2218-45ce-b082-64fe59f772a2-istiod-ca-cert" pod "886729d6-2218-45ce-b082-64fe59f772a2" (UID: "886729d6-2218-45ce-b082-64fe59f772a2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:13:15.600738 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.600619 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-workload-certs" pod "886729d6-2218-45ce-b082-64fe59f772a2" (UID: "886729d6-2218-45ce-b082-64fe59f772a2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:15.601086 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.601062 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:15.601169 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.601092 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:15.601169 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.601108 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:15.601169 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.601125 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/886729d6-2218-45ce-b082-64fe59f772a2-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:15.601169 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.601139 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:15.601954 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.601925 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/886729d6-2218-45ce-b082-64fe59f772a2-istio-podinfo" pod "886729d6-2218-45ce-b082-64fe59f772a2" (UID: "886729d6-2218-45ce-b082-64fe59f772a2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:13:15.602099 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.602082 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-istio-envoy" pod "886729d6-2218-45ce-b082-64fe59f772a2" (UID: "886729d6-2218-45ce-b082-64fe59f772a2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:15.602279 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.602265 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/886729d6-2218-45ce-b082-64fe59f772a2-kube-api-access-sps6v" pod "886729d6-2218-45ce-b082-64fe59f772a2" (UID: "886729d6-2218-45ce-b082-64fe59f772a2"). InnerVolumeSpecName "kube-api-access-sps6v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:13:15.602381 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.602367 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/886729d6-2218-45ce-b082-64fe59f772a2-istio-token" pod "886729d6-2218-45ce-b082-64fe59f772a2" (UID: "886729d6-2218-45ce-b082-64fe59f772a2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:13:15.602878 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.602863 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:13:15.711058 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.704369 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istiod-ca-cert\") pod \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " Oct 30 19:13:15.711058 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.704864 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istiod-ca-cert" pod "5a3f5720-0f19-4f6d-9609-1924e6d45b3e" (UID: "5a3f5720-0f19-4f6d-9609-1924e6d45b3e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:13:15.711058 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.704994 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-kube-api-access-pvsmc\" (UniqueName: \"kubernetes.io/projected/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-kube-api-access-pvsmc\") pod \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " Oct 30 19:13:15.711058 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.705066 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-envoy\") pod \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " Oct 30 19:13:15.711058 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.709704 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-credential-socket" pod "5a3f5720-0f19-4f6d-9609-1924e6d45b3e" (UID: "5a3f5720-0f19-4f6d-9609-1924e6d45b3e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:15.711058 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.709812 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-credential-socket\") pod \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " Oct 30 19:13:15.711058 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.710095 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-workload-certs" pod "5a3f5720-0f19-4f6d-9609-1924e6d45b3e" (UID: "5a3f5720-0f19-4f6d-9609-1924e6d45b3e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:15.711058 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.709865 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-workload-certs\") pod \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " Oct 30 19:13:15.711058 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.710231 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-workload-socket\") pod \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " Oct 30 19:13:15.712726 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.712671 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-workload-socket" pod "5a3f5720-0f19-4f6d-9609-1924e6d45b3e" (UID: "5a3f5720-0f19-4f6d-9609-1924e6d45b3e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:15.713787 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.713207 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-podinfo\") pod \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " Oct 30 19:13:15.713787 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.713717 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-data\") pod \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " Oct 30 19:13:15.727676 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.727630 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-data" pod "5a3f5720-0f19-4f6d-9609-1924e6d45b3e" (UID: "5a3f5720-0f19-4f6d-9609-1924e6d45b3e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:15.727943 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.713754 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-token\" (UniqueName: \"kubernetes.io/projected/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-token\") pod \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\" (UID: \"5a3f5720-0f19-4f6d-9609-1924e6d45b3e\") " Oct 30 19:13:15.729566 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.729487 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:15.729836 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.729779 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/886729d6-2218-45ce-b082-64fe59f772a2-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:15.729836 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.729804 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sps6v\" (UniqueName: \"kubernetes.io/projected/886729d6-2218-45ce-b082-64fe59f772a2-kube-api-access-sps6v\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:15.730152 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.730074 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:15.730152 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.730102 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/886729d6-2218-45ce-b082-64fe59f772a2-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:15.730152 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.730118 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/886729d6-2218-45ce-b082-64fe59f772a2-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:15.730654 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.730132 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:15.730868 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.730825 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:15.730868 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.730847 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:15.745661 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.744780 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-kube-api-access-pvsmc" pod "5a3f5720-0f19-4f6d-9609-1924e6d45b3e" (UID: "5a3f5720-0f19-4f6d-9609-1924e6d45b3e"). InnerVolumeSpecName "kube-api-access-pvsmc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:13:15.757033 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.756832 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-podinfo" pod "5a3f5720-0f19-4f6d-9609-1924e6d45b3e" (UID: "5a3f5720-0f19-4f6d-9609-1924e6d45b3e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:13:15.757033 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.756990 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-envoy" pod "5a3f5720-0f19-4f6d-9609-1924e6d45b3e" (UID: "5a3f5720-0f19-4f6d-9609-1924e6d45b3e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:15.802820 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.802760 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-token" pod "5a3f5720-0f19-4f6d-9609-1924e6d45b3e" (UID: "5a3f5720-0f19-4f6d-9609-1924e6d45b3e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:13:15.835966 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.835239 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvsmc\" (UniqueName: \"kubernetes.io/projected/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-kube-api-access-pvsmc\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:15.835966 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.835277 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:15.835966 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.835289 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:15.835966 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:15.835301 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a3f5720-0f19-4f6d-9609-1924e6d45b3e-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.049155 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:13:16.049102 5894 cadvisor_stats_provider.go:569] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48ae9925_3d54_48a5_af7a_d6bc186ea365.slice/crio-conmon-b0a98eee3b434de1693b0eee3a355b5bad53d4cddcb4268e8f5687abff3fc7b7.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43b1379e_42d3_4913_ab29_d766689a4a90.slice/crio-conmon-469a6bab9d326996c01f9c9d782beebc8e837f38ff66edadf4f948d0931b05ba.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod43b1379e_42d3_4913_ab29_d766689a4a90.slice/crio-469a6bab9d326996c01f9c9d782beebc8e837f38ff66edadf4f948d0931b05ba.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48ae9925_3d54_48a5_af7a_d6bc186ea365.slice/crio-b0a98eee3b434de1693b0eee3a355b5bad53d4cddcb4268e8f5687abff3fc7b7.scope\": RecentStats: unable to find data in memory cache]" Oct 30 19:13:16.141820 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.141669 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:13:16.183524 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.183479 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:16.248708 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.248623 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-workload-socket\") pod \"11f90211-60b6-41d7-a44f-45198e9eb898\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " Oct 30 19:13:16.248708 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.248683 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/11f90211-60b6-41d7-a44f-45198e9eb898-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11f90211-60b6-41d7-a44f-45198e9eb898-istio-podinfo\") pod \"11f90211-60b6-41d7-a44f-45198e9eb898\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " Oct 30 19:13:16.248916 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.248769 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-credential-socket\") pod \"11f90211-60b6-41d7-a44f-45198e9eb898\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " Oct 30 19:13:16.248916 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.248824 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11f90211-60b6-41d7-a44f-45198e9eb898-kube-api-access-snz24\" (UniqueName: \"kubernetes.io/projected/11f90211-60b6-41d7-a44f-45198e9eb898-kube-api-access-snz24\") pod \"11f90211-60b6-41d7-a44f-45198e9eb898\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " Oct 30 19:13:16.248916 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.248857 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-istio-data\") pod \"11f90211-60b6-41d7-a44f-45198e9eb898\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " Oct 30 19:13:16.248916 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.248904 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/11f90211-60b6-41d7-a44f-45198e9eb898-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11f90211-60b6-41d7-a44f-45198e9eb898-istiod-ca-cert\") pod \"11f90211-60b6-41d7-a44f-45198e9eb898\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " Oct 30 19:13:16.249147 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.248993 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/11f90211-60b6-41d7-a44f-45198e9eb898-istio-token\" (UniqueName: \"kubernetes.io/projected/11f90211-60b6-41d7-a44f-45198e9eb898-istio-token\") pod \"11f90211-60b6-41d7-a44f-45198e9eb898\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " Oct 30 19:13:16.249147 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.249125 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-istio-envoy\") pod \"11f90211-60b6-41d7-a44f-45198e9eb898\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " Oct 30 19:13:16.249254 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.249203 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-workload-certs\") pod \"11f90211-60b6-41d7-a44f-45198e9eb898\" (UID: \"11f90211-60b6-41d7-a44f-45198e9eb898\") " Oct 30 19:13:16.250799 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.250768 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-workload-certs" pod "11f90211-60b6-41d7-a44f-45198e9eb898" (UID: "11f90211-60b6-41d7-a44f-45198e9eb898"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:16.251006 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.250986 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-istio-data" pod "11f90211-60b6-41d7-a44f-45198e9eb898" (UID: "11f90211-60b6-41d7-a44f-45198e9eb898"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:16.251315 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.251293 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11f90211-60b6-41d7-a44f-45198e9eb898-istiod-ca-cert" pod "11f90211-60b6-41d7-a44f-45198e9eb898" (UID: "11f90211-60b6-41d7-a44f-45198e9eb898"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:13:16.252420 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.252381 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-workload-socket" pod "11f90211-60b6-41d7-a44f-45198e9eb898" (UID: "11f90211-60b6-41d7-a44f-45198e9eb898"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:16.252727 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.252705 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-credential-socket" pod "11f90211-60b6-41d7-a44f-45198e9eb898" (UID: "11f90211-60b6-41d7-a44f-45198e9eb898"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:16.254125 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.254097 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11f90211-60b6-41d7-a44f-45198e9eb898-istio-token" pod "11f90211-60b6-41d7-a44f-45198e9eb898" (UID: "11f90211-60b6-41d7-a44f-45198e9eb898"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:13:16.276508 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.276454 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/11f90211-60b6-41d7-a44f-45198e9eb898-istio-podinfo" pod "11f90211-60b6-41d7-a44f-45198e9eb898" (UID: "11f90211-60b6-41d7-a44f-45198e9eb898"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:13:16.277032 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.276946 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11f90211-60b6-41d7-a44f-45198e9eb898-kube-api-access-snz24" pod "11f90211-60b6-41d7-a44f-45198e9eb898" (UID: "11f90211-60b6-41d7-a44f-45198e9eb898"). InnerVolumeSpecName "kube-api-access-snz24". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:13:16.277032 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.276947 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-istio-envoy" pod "11f90211-60b6-41d7-a44f-45198e9eb898" (UID: "11f90211-60b6-41d7-a44f-45198e9eb898"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:16.327093 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.327063 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 19:13:16.329498 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.329425 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:13:16.351435 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.351399 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/11f90211-60b6-41d7-a44f-45198e9eb898-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.351567 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.351440 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/11f90211-60b6-41d7-a44f-45198e9eb898-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.351567 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.351456 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.351567 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.351470 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.351567 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.351486 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.351567 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.351502 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/11f90211-60b6-41d7-a44f-45198e9eb898-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.351567 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.351516 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.351567 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.351529 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-snz24\" (UniqueName: \"kubernetes.io/projected/11f90211-60b6-41d7-a44f-45198e9eb898-kube-api-access-snz24\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.351567 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.351543 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/11f90211-60b6-41d7-a44f-45198e9eb898-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.402568 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.402538 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" event={"ID":"5a3f5720-0f19-4f6d-9609-1924e6d45b3e","Type":"ContainerDied","Data":"81c64e7ba1677b1e63ff319493661afbf72ca51d936063f22564dfd9f729ee81"} Oct 30 19:13:16.402753 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.402546 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg" Oct 30 19:13:16.402753 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.402624 5894 scope.go:122] "RemoveContainer" containerID="fd5be8b365afbc1c2379b5192a44abe351aeae024d322abd470817ae5ba61023" Oct 30 19:13:16.404200 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.404178 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" Oct 30 19:13:16.404330 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.404215 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69" event={"ID":"886729d6-2218-45ce-b082-64fe59f772a2","Type":"ContainerDied","Data":"490227e12be4afa0add2d2a2900cd0d09506b5b943508f608f2e5ffa907f6ba1"} Oct 30 19:13:16.406118 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.406065 5894 generic.go:356] "Generic (PLEG): container finished" podID="48ae9925-3d54-48a5-af7a-d6bc186ea365" containerID="b0a98eee3b434de1693b0eee3a355b5bad53d4cddcb4268e8f5687abff3fc7b7" exitCode=0 Oct 30 19:13:16.406239 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.406121 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" Oct 30 19:13:16.406239 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.406147 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" event={"ID":"48ae9925-3d54-48a5-af7a-d6bc186ea365","Type":"ContainerDied","Data":"b0a98eee3b434de1693b0eee3a355b5bad53d4cddcb4268e8f5687abff3fc7b7"} Oct 30 19:13:16.406239 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.406190 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh" event={"ID":"48ae9925-3d54-48a5-af7a-d6bc186ea365","Type":"ContainerDied","Data":"3e300ee09ac146c780c5b73b7d0dfdb01d57b18e503bc3208807580be70ab0d9"} Oct 30 19:13:16.409410 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.409388 5894 generic.go:356] "Generic (PLEG): container finished" podID="11f90211-60b6-41d7-a44f-45198e9eb898" containerID="a3021eab31d5bdaff70b125da4f2f54a46fe04d5694027883eb2ca42e689b4ac" exitCode=0 Oct 30 19:13:16.409516 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.409423 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" Oct 30 19:13:16.409516 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.409453 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" event={"ID":"11f90211-60b6-41d7-a44f-45198e9eb898","Type":"ContainerDied","Data":"a3021eab31d5bdaff70b125da4f2f54a46fe04d5694027883eb2ca42e689b4ac"} Oct 30 19:13:16.409516 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.409499 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs" event={"ID":"11f90211-60b6-41d7-a44f-45198e9eb898","Type":"ContainerDied","Data":"69f3cd10340be3bc4d8722b16b0a630aa9bab872845235d2f18d6c3333031454"} Oct 30 19:13:16.410964 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.410939 5894 generic.go:356] "Generic (PLEG): container finished" podID="43b1379e-42d3-4913-ab29-d766689a4a90" containerID="469a6bab9d326996c01f9c9d782beebc8e837f38ff66edadf4f948d0931b05ba" exitCode=0 Oct 30 19:13:16.411070 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.410973 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" Oct 30 19:13:16.411070 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.411027 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" event={"ID":"43b1379e-42d3-4913-ab29-d766689a4a90","Type":"ContainerDied","Data":"469a6bab9d326996c01f9c9d782beebc8e837f38ff66edadf4f948d0931b05ba"} Oct 30 19:13:16.411165 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.411075 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq" event={"ID":"43b1379e-42d3-4913-ab29-d766689a4a90","Type":"ContainerDied","Data":"9cb286d77315791d277506e77344cc2a32def491fb0ff80ee056e3f13402d969"} Oct 30 19:13:16.414635 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.414617 5894 scope.go:122] "RemoveContainer" containerID="8b052e8b22a76b049786fe3e5a915cf20450d5ed3552371e8f4f82700e535bf9" Oct 30 19:13:16.423888 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.423873 5894 scope.go:122] "RemoveContainer" containerID="b0a98eee3b434de1693b0eee3a355b5bad53d4cddcb4268e8f5687abff3fc7b7" Oct 30 19:13:16.433935 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.433920 5894 scope.go:122] "RemoveContainer" containerID="b0a98eee3b434de1693b0eee3a355b5bad53d4cddcb4268e8f5687abff3fc7b7" Oct 30 19:13:16.434207 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:13:16.434189 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b0a98eee3b434de1693b0eee3a355b5bad53d4cddcb4268e8f5687abff3fc7b7\": container with ID starting with b0a98eee3b434de1693b0eee3a355b5bad53d4cddcb4268e8f5687abff3fc7b7 not found: ID does not exist" containerID="b0a98eee3b434de1693b0eee3a355b5bad53d4cddcb4268e8f5687abff3fc7b7" Oct 30 19:13:16.434283 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.434214 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b0a98eee3b434de1693b0eee3a355b5bad53d4cddcb4268e8f5687abff3fc7b7"} err="failed to get container status \"b0a98eee3b434de1693b0eee3a355b5bad53d4cddcb4268e8f5687abff3fc7b7\": rpc error: code = NotFound desc = could not find container \"b0a98eee3b434de1693b0eee3a355b5bad53d4cddcb4268e8f5687abff3fc7b7\": container with ID starting with b0a98eee3b434de1693b0eee3a355b5bad53d4cddcb4268e8f5687abff3fc7b7 not found: ID does not exist" Oct 30 19:13:16.434283 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.434231 5894 scope.go:122] "RemoveContainer" containerID="a3021eab31d5bdaff70b125da4f2f54a46fe04d5694027883eb2ca42e689b4ac" Oct 30 19:13:16.450283 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.450255 5894 scope.go:122] "RemoveContainer" containerID="a3021eab31d5bdaff70b125da4f2f54a46fe04d5694027883eb2ca42e689b4ac" Oct 30 19:13:16.450778 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:13:16.450699 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a3021eab31d5bdaff70b125da4f2f54a46fe04d5694027883eb2ca42e689b4ac\": container with ID starting with a3021eab31d5bdaff70b125da4f2f54a46fe04d5694027883eb2ca42e689b4ac not found: ID does not exist" containerID="a3021eab31d5bdaff70b125da4f2f54a46fe04d5694027883eb2ca42e689b4ac" Oct 30 19:13:16.450778 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.450736 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a3021eab31d5bdaff70b125da4f2f54a46fe04d5694027883eb2ca42e689b4ac"} err="failed to get container status \"a3021eab31d5bdaff70b125da4f2f54a46fe04d5694027883eb2ca42e689b4ac\": rpc error: code = NotFound desc = could not find container \"a3021eab31d5bdaff70b125da4f2f54a46fe04d5694027883eb2ca42e689b4ac\": container with ID starting with a3021eab31d5bdaff70b125da4f2f54a46fe04d5694027883eb2ca42e689b4ac not found: ID does not exist" Oct 30 19:13:16.450778 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.450760 5894 scope.go:122] "RemoveContainer" containerID="469a6bab9d326996c01f9c9d782beebc8e837f38ff66edadf4f948d0931b05ba" Oct 30 19:13:16.452769 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.452741 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-envoy\") pod \"48ae9925-3d54-48a5-af7a-d6bc186ea365\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " Oct 30 19:13:16.453996 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.453959 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-workload-socket" pod "48ae9925-3d54-48a5-af7a-d6bc186ea365" (UID: "48ae9925-3d54-48a5-af7a-d6bc186ea365"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:16.454124 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.454008 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-workload-socket\") pod \"48ae9925-3d54-48a5-af7a-d6bc186ea365\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " Oct 30 19:13:16.454124 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.454094 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-workload-certs\") pod \"48ae9925-3d54-48a5-af7a-d6bc186ea365\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " Oct 30 19:13:16.454400 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.454366 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-workload-certs" pod "48ae9925-3d54-48a5-af7a-d6bc186ea365" (UID: "48ae9925-3d54-48a5-af7a-d6bc186ea365"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:16.454498 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.454421 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-istio-envoy\") pod \"43b1379e-42d3-4913-ab29-d766689a4a90\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " Oct 30 19:13:16.454626 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.454517 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-token\" (UniqueName: \"kubernetes.io/projected/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-token\") pod \"48ae9925-3d54-48a5-af7a-d6bc186ea365\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " Oct 30 19:13:16.454626 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.454560 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/43b1379e-42d3-4913-ab29-d766689a4a90-kube-api-access-xzlxz\" (UniqueName: \"kubernetes.io/projected/43b1379e-42d3-4913-ab29-d766689a4a90-kube-api-access-xzlxz\") pod \"43b1379e-42d3-4913-ab29-d766689a4a90\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " Oct 30 19:13:16.454626 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.454612 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-credential-socket\") pod \"43b1379e-42d3-4913-ab29-d766689a4a90\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " Oct 30 19:13:16.454830 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.454660 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48ae9925-3d54-48a5-af7a-d6bc186ea365-kube-api-access-2npqx\" (UniqueName: \"kubernetes.io/projected/48ae9925-3d54-48a5-af7a-d6bc186ea365-kube-api-access-2npqx\") pod \"48ae9925-3d54-48a5-af7a-d6bc186ea365\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " Oct 30 19:13:16.454830 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.454718 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-workload-socket\") pod \"43b1379e-42d3-4913-ab29-d766689a4a90\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " Oct 30 19:13:16.454830 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.454776 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/43b1379e-42d3-4913-ab29-d766689a4a90-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43b1379e-42d3-4913-ab29-d766689a4a90-istio-podinfo\") pod \"43b1379e-42d3-4913-ab29-d766689a4a90\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " Oct 30 19:13:16.454830 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.454818 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-credential-socket\") pod \"48ae9925-3d54-48a5-af7a-d6bc186ea365\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " Oct 30 19:13:16.455066 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.454931 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/48ae9925-3d54-48a5-af7a-d6bc186ea365-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48ae9925-3d54-48a5-af7a-d6bc186ea365-istiod-ca-cert\") pod \"48ae9925-3d54-48a5-af7a-d6bc186ea365\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " Oct 30 19:13:16.455066 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.454957 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-data\") pod \"48ae9925-3d54-48a5-af7a-d6bc186ea365\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " Oct 30 19:13:16.455066 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.454994 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-workload-certs\") pod \"43b1379e-42d3-4913-ab29-d766689a4a90\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " Oct 30 19:13:16.455066 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.455047 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-podinfo\") pod \"48ae9925-3d54-48a5-af7a-d6bc186ea365\" (UID: \"48ae9925-3d54-48a5-af7a-d6bc186ea365\") " Oct 30 19:13:16.455753 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.455202 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/43b1379e-42d3-4913-ab29-d766689a4a90-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43b1379e-42d3-4913-ab29-d766689a4a90-istiod-ca-cert\") pod \"43b1379e-42d3-4913-ab29-d766689a4a90\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " Oct 30 19:13:16.455753 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.455314 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/43b1379e-42d3-4913-ab29-d766689a4a90-istio-token\" (UniqueName: \"kubernetes.io/projected/43b1379e-42d3-4913-ab29-d766689a4a90-istio-token\") pod \"43b1379e-42d3-4913-ab29-d766689a4a90\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " Oct 30 19:13:16.455753 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.455389 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-istio-data\") pod \"43b1379e-42d3-4913-ab29-d766689a4a90\" (UID: \"43b1379e-42d3-4913-ab29-d766689a4a90\") " Oct 30 19:13:16.459957 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.458130 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.459957 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.458159 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.459957 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.458357 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-istio-data" pod "43b1379e-42d3-4913-ab29-d766689a4a90" (UID: "43b1379e-42d3-4913-ab29-d766689a4a90"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:16.459957 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.458563 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-credential-socket" pod "43b1379e-42d3-4913-ab29-d766689a4a90" (UID: "43b1379e-42d3-4913-ab29-d766689a4a90"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:16.459957 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.459434 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-data" pod "48ae9925-3d54-48a5-af7a-d6bc186ea365" (UID: "48ae9925-3d54-48a5-af7a-d6bc186ea365"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:16.459957 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.459647 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-workload-socket" pod "43b1379e-42d3-4913-ab29-d766689a4a90" (UID: "43b1379e-42d3-4913-ab29-d766689a4a90"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:16.468087 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.464483 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-workload-certs" pod "43b1379e-42d3-4913-ab29-d766689a4a90" (UID: "43b1379e-42d3-4913-ab29-d766689a4a90"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:16.468087 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.465098 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43b1379e-42d3-4913-ab29-d766689a4a90-istiod-ca-cert" pod "43b1379e-42d3-4913-ab29-d766689a4a90" (UID: "43b1379e-42d3-4913-ab29-d766689a4a90"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:13:16.468087 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.465211 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48ae9925-3d54-48a5-af7a-d6bc186ea365-kube-api-access-2npqx" pod "48ae9925-3d54-48a5-af7a-d6bc186ea365" (UID: "48ae9925-3d54-48a5-af7a-d6bc186ea365"). InnerVolumeSpecName "kube-api-access-2npqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:13:16.468087 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.466067 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/48ae9925-3d54-48a5-af7a-d6bc186ea365-istiod-ca-cert" pod "48ae9925-3d54-48a5-af7a-d6bc186ea365" (UID: "48ae9925-3d54-48a5-af7a-d6bc186ea365"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:13:16.468087 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.466151 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-credential-socket" pod "48ae9925-3d54-48a5-af7a-d6bc186ea365" (UID: "48ae9925-3d54-48a5-af7a-d6bc186ea365"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:16.468645 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.468556 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-envoy" pod "48ae9925-3d54-48a5-af7a-d6bc186ea365" (UID: "48ae9925-3d54-48a5-af7a-d6bc186ea365"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:16.475299 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.475258 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-podinfo" pod "48ae9925-3d54-48a5-af7a-d6bc186ea365" (UID: "48ae9925-3d54-48a5-af7a-d6bc186ea365"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:13:16.475299 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.475280 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-token" pod "48ae9925-3d54-48a5-af7a-d6bc186ea365" (UID: "48ae9925-3d54-48a5-af7a-d6bc186ea365"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:13:16.475453 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.475330 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43b1379e-42d3-4913-ab29-d766689a4a90-kube-api-access-xzlxz" pod "43b1379e-42d3-4913-ab29-d766689a4a90" (UID: "43b1379e-42d3-4913-ab29-d766689a4a90"). InnerVolumeSpecName "kube-api-access-xzlxz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:13:16.475643 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.475565 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-istio-envoy" pod "43b1379e-42d3-4913-ab29-d766689a4a90" (UID: "43b1379e-42d3-4913-ab29-d766689a4a90"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:16.475643 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.475612 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43b1379e-42d3-4913-ab29-d766689a4a90-istio-token" pod "43b1379e-42d3-4913-ab29-d766689a4a90" (UID: "43b1379e-42d3-4913-ab29-d766689a4a90"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:13:16.482183 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.482159 5894 scope.go:122] "RemoveContainer" containerID="469a6bab9d326996c01f9c9d782beebc8e837f38ff66edadf4f948d0931b05ba" Oct 30 19:13:16.482533 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:13:16.482472 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"469a6bab9d326996c01f9c9d782beebc8e837f38ff66edadf4f948d0931b05ba\": container with ID starting with 469a6bab9d326996c01f9c9d782beebc8e837f38ff66edadf4f948d0931b05ba not found: ID does not exist" containerID="469a6bab9d326996c01f9c9d782beebc8e837f38ff66edadf4f948d0931b05ba" Oct 30 19:13:16.482533 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.482508 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"469a6bab9d326996c01f9c9d782beebc8e837f38ff66edadf4f948d0931b05ba"} err="failed to get container status \"469a6bab9d326996c01f9c9d782beebc8e837f38ff66edadf4f948d0931b05ba\": rpc error: code = NotFound desc = could not find container \"469a6bab9d326996c01f9c9d782beebc8e837f38ff66edadf4f948d0931b05ba\": container with ID starting with 469a6bab9d326996c01f9c9d782beebc8e837f38ff66edadf4f948d0931b05ba not found: ID does not exist" Oct 30 19:13:16.490704 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.490682 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/43b1379e-42d3-4913-ab29-d766689a4a90-istio-podinfo" pod "43b1379e-42d3-4913-ab29-d766689a4a90" (UID: "43b1379e-42d3-4913-ab29-d766689a4a90"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:13:16.565634 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.565487 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/43b1379e-42d3-4913-ab29-d766689a4a90-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.565634 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.565522 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.565634 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.565533 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.565634 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.565544 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.565634 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.565557 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.565634 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.565594 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xzlxz\" (UniqueName: \"kubernetes.io/projected/43b1379e-42d3-4913-ab29-d766689a4a90-kube-api-access-xzlxz\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.565634 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.565608 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.565634 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.565631 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2npqx\" (UniqueName: \"kubernetes.io/projected/48ae9925-3d54-48a5-af7a-d6bc186ea365-kube-api-access-2npqx\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.566212 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.565649 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.566212 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.565668 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/43b1379e-42d3-4913-ab29-d766689a4a90-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.566212 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.565686 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.566212 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.565704 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/48ae9925-3d54-48a5-af7a-d6bc186ea365-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.566212 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.565719 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.566212 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.565731 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/43b1379e-42d3-4913-ab29-d766689a4a90-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.566212 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.565761 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/48ae9925-3d54-48a5-af7a-d6bc186ea365-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.566212 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.565774 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/43b1379e-42d3-4913-ab29-d766689a4a90-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:16.600707 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.600675 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:16.746704 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.746671 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69"] Oct 30 19:13:16.749994 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.749971 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tvj69"] Oct 30 19:13:16.853971 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.853890 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs"] Oct 30 19:13:16.859605 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.859561 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-tv8zs"] Oct 30 19:13:16.880409 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.880375 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:16.976381 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.976345 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg"] Oct 30 19:13:16.980672 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:16.980646 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-fpdvg"] Oct 30 19:13:17.091543 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:17.091511 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq"] Oct 30 19:13:17.095349 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:17.095320 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-d58sq"] Oct 30 19:13:17.172603 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:17.169666 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:17.232750 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:17.232673 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh"] Oct 30 19:13:17.238366 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:17.237898 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-486bh"] Oct 30 19:13:17.651363 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:17.651332 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="11f90211-60b6-41d7-a44f-45198e9eb898" path="/var/lib/kubelet/pods/11f90211-60b6-41d7-a44f-45198e9eb898/volumes" Oct 30 19:13:17.651853 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:17.651839 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="43b1379e-42d3-4913-ab29-d766689a4a90" path="/var/lib/kubelet/pods/43b1379e-42d3-4913-ab29-d766689a4a90/volumes" Oct 30 19:13:17.652284 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:17.652271 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="48ae9925-3d54-48a5-af7a-d6bc186ea365" path="/var/lib/kubelet/pods/48ae9925-3d54-48a5-af7a-d6bc186ea365/volumes" Oct 30 19:13:17.653074 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:17.653059 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5a3f5720-0f19-4f6d-9609-1924e6d45b3e" path="/var/lib/kubelet/pods/5a3f5720-0f19-4f6d-9609-1924e6d45b3e/volumes" Oct 30 19:13:17.653535 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:17.653521 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="886729d6-2218-45ce-b082-64fe59f772a2" path="/var/lib/kubelet/pods/886729d6-2218-45ce-b082-64fe59f772a2/volumes" Oct 30 19:13:20.119000 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:13:20.118966 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:13:21.192795 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:21.192762 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:22.198386 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:22.190297 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:22.388558 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:22.388510 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:22.598619 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:22.598465 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:22.833665 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:22.833627 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:23.039159 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:23.039110 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:23.619623 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:23.619568 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:25.179622 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:25.179588 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:25.518399 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:25.518212 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:25.814654 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:25.804781 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:25.886252 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:25.886203 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr"] Oct 30 19:13:25.886440 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:25.886394 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" podUID="bcb8e536-b0a5-4517-8c98-a39793531709" containerName="istio-proxy" containerID="cri-o://ffe0b104ba9ba05a9a318fc35f18cefa7e5059cfafc4362b737fa14e467441f3" gracePeriod=30 Oct 30 19:13:26.570485 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.570449 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4"] Oct 30 19:13:26.571201 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.571184 5894 cpu_manager.go:414] "containerMap: removing container" podUID="43b1379e-42d3-4913-ab29-d766689a4a90" containerName="istio-proxy" Oct 30 19:13:26.571269 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.571202 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="43b1379e-42d3-4913-ab29-d766689a4a90" containerName="istio-proxy" Oct 30 19:13:26.571269 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.571217 5894 cpu_manager.go:414] "containerMap: removing container" podUID="5a3f5720-0f19-4f6d-9609-1924e6d45b3e" containerName="istio-proxy" Oct 30 19:13:26.571269 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.571222 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5a3f5720-0f19-4f6d-9609-1924e6d45b3e" containerName="istio-proxy" Oct 30 19:13:26.571269 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.571242 5894 cpu_manager.go:414] "containerMap: removing container" podUID="48ae9925-3d54-48a5-af7a-d6bc186ea365" containerName="istio-proxy" Oct 30 19:13:26.571269 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.571248 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48ae9925-3d54-48a5-af7a-d6bc186ea365" containerName="istio-proxy" Oct 30 19:13:26.571269 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.571262 5894 cpu_manager.go:414] "containerMap: removing container" podUID="886729d6-2218-45ce-b082-64fe59f772a2" containerName="istio-proxy" Oct 30 19:13:26.571269 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.571267 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="886729d6-2218-45ce-b082-64fe59f772a2" containerName="istio-proxy" Oct 30 19:13:26.571610 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.571284 5894 cpu_manager.go:414] "containerMap: removing container" podUID="11f90211-60b6-41d7-a44f-45198e9eb898" containerName="istio-proxy" Oct 30 19:13:26.571610 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.571292 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="11f90211-60b6-41d7-a44f-45198e9eb898" containerName="istio-proxy" Oct 30 19:13:26.571610 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.571425 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="43b1379e-42d3-4913-ab29-d766689a4a90" containerName="istio-proxy" Oct 30 19:13:26.571610 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.571435 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="5a3f5720-0f19-4f6d-9609-1924e6d45b3e" containerName="istio-proxy" Oct 30 19:13:26.571610 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.571443 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="48ae9925-3d54-48a5-af7a-d6bc186ea365" containerName="istio-proxy" Oct 30 19:13:26.571610 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.571450 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="886729d6-2218-45ce-b082-64fe59f772a2" containerName="istio-proxy" Oct 30 19:13:26.571610 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.571456 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="11f90211-60b6-41d7-a44f-45198e9eb898" containerName="istio-proxy" Oct 30 19:13:26.576467 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.576444 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.632966 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.632931 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-e0fc1f-openshift-default-dockercfg-wcqfh\"" Oct 30 19:13:26.701618 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.701569 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4"] Oct 30 19:13:26.721841 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.721799 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-istio-data\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.725701 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.725669 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0479234-1db6-465f-9454-54d777b09538-istio-podinfo\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.726050 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.726023 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-credential-socket\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.726213 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.726196 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-workload-socket\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.726360 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.726344 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-istio-envoy\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.726525 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.726506 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wg5r8\" (UniqueName: \"kubernetes.io/projected/b0479234-1db6-465f-9454-54d777b09538-kube-api-access-wg5r8\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.726697 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.726676 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-workload-certs\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.733596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.726902 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0479234-1db6-465f-9454-54d777b09538-istio-token\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.733895 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.733866 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0479234-1db6-465f-9454-54d777b09538-istiod-ca-cert\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.734079 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.734056 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:26.836310 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.836210 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-credential-socket\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.836733 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.836710 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-workload-socket\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.836906 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.836885 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-istio-envoy\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.837084 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.837064 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wg5r8\" (UniqueName: \"kubernetes.io/projected/b0479234-1db6-465f-9454-54d777b09538-kube-api-access-wg5r8\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.837639 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.837616 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-workload-certs\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.837809 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.837788 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0479234-1db6-465f-9454-54d777b09538-istio-token\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.837986 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.837644 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-workload-socket\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.837986 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.837970 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0479234-1db6-465f-9454-54d777b09538-istiod-ca-cert\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.838463 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.838436 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-istio-data\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.838546 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.838496 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0479234-1db6-465f-9454-54d777b09538-istio-podinfo\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.838930 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.838910 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-istio-data\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.839443 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.836989 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-credential-socket\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.839557 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.839390 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0479234-1db6-465f-9454-54d777b09538-istiod-ca-cert\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.839738 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.839711 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-workload-certs\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.898108 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.898042 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wg5r8\" (UniqueName: \"kubernetes.io/projected/b0479234-1db6-465f-9454-54d777b09538-kube-api-access-wg5r8\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.900079 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.900050 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0479234-1db6-465f-9454-54d777b09538-istio-token\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.909872 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.909807 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-istio-envoy\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.910923 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.910844 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0479234-1db6-465f-9454-54d777b09538-istio-podinfo\") pod \"e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:26.914380 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:26.914356 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:27.193030 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:27.192986 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:27.472176 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:27.468776 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:27.652142 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:27.652111 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4"] Oct 30 19:13:27.687682 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:13:27.687655 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0479234_1db6_465f_9454_54d777b09538.slice/crio-624ae13ac7fe2da5306a2a5737608147a36c6c13215d1e8a6dafceb93b5700db WatchSource:0}: Error finding container 624ae13ac7fe2da5306a2a5737608147a36c6c13215d1e8a6dafceb93b5700db: Status 404 returned error can't find the container with id 624ae13ac7fe2da5306a2a5737608147a36c6c13215d1e8a6dafceb93b5700db Oct 30 19:13:27.690736 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:27.690697 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:13:27.690823 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:27.690783 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:13:27.690867 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:27.690828 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:13:27.808274 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:27.808239 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:28.017655 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:28.017522 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:28.624090 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:28.624050 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" event={"ID":"b0479234-1db6-465f-9454-54d777b09538","Type":"ContainerStarted","Data":"cdc0d40f50f26e4e40419f58a5a929522a0e258cae6deefd5c7127e887a48cc7"} Oct 30 19:13:28.624264 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:28.624112 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" event={"ID":"b0479234-1db6-465f-9454-54d777b09538","Type":"ContainerStarted","Data":"624ae13ac7fe2da5306a2a5737608147a36c6c13215d1e8a6dafceb93b5700db"} Oct 30 19:13:28.827939 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:28.827878 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" podStartSLOduration=2.827866369 podStartE2EDuration="2.827866369s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:13:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:13:28.792777912 +0000 UTC m=+2079.282905753" watchObservedRunningTime="2026-10-30 19:13:28.827866369 +0000 UTC m=+2079.317994189" Oct 30 19:13:29.004773 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:29.004706 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:29.187482 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:29.187451 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:29.193438 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:29.193409 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:29.199416 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:29.199394 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:29.630128 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:29.630094 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:29.631173 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:29.631154 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:13:29.903054 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:29.903020 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:13:29.903536 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:29.903078 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" podUID="bcb8e536-b0a5-4517-8c98-a39793531709" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:13:31.188774 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.188746 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:13:31.245332 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.245303 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bcb8e536-b0a5-4517-8c98-a39793531709-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcb8e536-b0a5-4517-8c98-a39793531709-istiod-ca-cert\") pod \"bcb8e536-b0a5-4517-8c98-a39793531709\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " Oct 30 19:13:31.245474 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.245368 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-workload-certs\") pod \"bcb8e536-b0a5-4517-8c98-a39793531709\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " Oct 30 19:13:31.245474 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.245393 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bcb8e536-b0a5-4517-8c98-a39793531709-istio-token\" (UniqueName: \"kubernetes.io/projected/bcb8e536-b0a5-4517-8c98-a39793531709-istio-token\") pod \"bcb8e536-b0a5-4517-8c98-a39793531709\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " Oct 30 19:13:31.245474 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.245440 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bcb8e536-b0a5-4517-8c98-a39793531709-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcb8e536-b0a5-4517-8c98-a39793531709-istio-podinfo\") pod \"bcb8e536-b0a5-4517-8c98-a39793531709\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " Oct 30 19:13:31.245665 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.245511 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-credential-socket\") pod \"bcb8e536-b0a5-4517-8c98-a39793531709\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " Oct 30 19:13:31.245665 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.245538 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-istio-data\") pod \"bcb8e536-b0a5-4517-8c98-a39793531709\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " Oct 30 19:13:31.245665 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.245570 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-workload-socket\") pod \"bcb8e536-b0a5-4517-8c98-a39793531709\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " Oct 30 19:13:31.245820 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.245725 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-workload-certs" pod "bcb8e536-b0a5-4517-8c98-a39793531709" (UID: "bcb8e536-b0a5-4517-8c98-a39793531709"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:31.245820 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.245741 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-istio-envoy\") pod \"bcb8e536-b0a5-4517-8c98-a39793531709\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " Oct 30 19:13:31.245820 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.245753 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcb8e536-b0a5-4517-8c98-a39793531709-istiod-ca-cert" pod "bcb8e536-b0a5-4517-8c98-a39793531709" (UID: "bcb8e536-b0a5-4517-8c98-a39793531709"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:13:31.245946 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.245866 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-credential-socket" pod "bcb8e536-b0a5-4517-8c98-a39793531709" (UID: "bcb8e536-b0a5-4517-8c98-a39793531709"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:31.245946 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.245879 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-istio-data" pod "bcb8e536-b0a5-4517-8c98-a39793531709" (UID: "bcb8e536-b0a5-4517-8c98-a39793531709"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:31.245946 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.245894 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-workload-socket" pod "bcb8e536-b0a5-4517-8c98-a39793531709" (UID: "bcb8e536-b0a5-4517-8c98-a39793531709"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:31.245946 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.245872 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bcb8e536-b0a5-4517-8c98-a39793531709-kube-api-access-vg49g\" (UniqueName: \"kubernetes.io/projected/bcb8e536-b0a5-4517-8c98-a39793531709-kube-api-access-vg49g\") pod \"bcb8e536-b0a5-4517-8c98-a39793531709\" (UID: \"bcb8e536-b0a5-4517-8c98-a39793531709\") " Oct 30 19:13:31.246438 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.246407 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:31.246551 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.246440 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:31.246551 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.246454 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:31.246551 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.246470 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bcb8e536-b0a5-4517-8c98-a39793531709-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:31.246551 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.246485 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:31.247964 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.247941 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcb8e536-b0a5-4517-8c98-a39793531709-kube-api-access-vg49g" pod "bcb8e536-b0a5-4517-8c98-a39793531709" (UID: "bcb8e536-b0a5-4517-8c98-a39793531709"). InnerVolumeSpecName "kube-api-access-vg49g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:13:31.247964 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.247945 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bcb8e536-b0a5-4517-8c98-a39793531709-istio-podinfo" pod "bcb8e536-b0a5-4517-8c98-a39793531709" (UID: "bcb8e536-b0a5-4517-8c98-a39793531709"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:13:31.248092 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.248002 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-istio-envoy" pod "bcb8e536-b0a5-4517-8c98-a39793531709" (UID: "bcb8e536-b0a5-4517-8c98-a39793531709"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:13:31.248092 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.248016 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcb8e536-b0a5-4517-8c98-a39793531709-istio-token" pod "bcb8e536-b0a5-4517-8c98-a39793531709" (UID: "bcb8e536-b0a5-4517-8c98-a39793531709"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:13:31.348011 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.347981 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcb8e536-b0a5-4517-8c98-a39793531709-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:31.348011 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.348006 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bcb8e536-b0a5-4517-8c98-a39793531709-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:31.348011 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.348015 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bcb8e536-b0a5-4517-8c98-a39793531709-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:31.348234 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.348024 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vg49g\" (UniqueName: \"kubernetes.io/projected/bcb8e536-b0a5-4517-8c98-a39793531709-kube-api-access-vg49g\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:31.589467 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.589373 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:31.645801 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.645769 5894 generic.go:356] "Generic (PLEG): container finished" podID="bcb8e536-b0a5-4517-8c98-a39793531709" containerID="ffe0b104ba9ba05a9a318fc35f18cefa7e5059cfafc4362b737fa14e467441f3" exitCode=0 Oct 30 19:13:31.646006 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.645866 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" Oct 30 19:13:31.654000 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.653971 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" event={"ID":"bcb8e536-b0a5-4517-8c98-a39793531709","Type":"ContainerDied","Data":"ffe0b104ba9ba05a9a318fc35f18cefa7e5059cfafc4362b737fa14e467441f3"} Oct 30 19:13:31.654149 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.654020 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr" event={"ID":"bcb8e536-b0a5-4517-8c98-a39793531709","Type":"ContainerDied","Data":"0c96e0c747c5d1129926c4ff396d22fa410679622257b475b3df13e5e219cd8c"} Oct 30 19:13:31.654149 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.654049 5894 scope.go:122] "RemoveContainer" containerID="ffe0b104ba9ba05a9a318fc35f18cefa7e5059cfafc4362b737fa14e467441f3" Oct 30 19:13:31.669774 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.669701 5894 scope.go:122] "RemoveContainer" containerID="ffe0b104ba9ba05a9a318fc35f18cefa7e5059cfafc4362b737fa14e467441f3" Oct 30 19:13:31.670059 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:13:31.670036 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ffe0b104ba9ba05a9a318fc35f18cefa7e5059cfafc4362b737fa14e467441f3\": container with ID starting with ffe0b104ba9ba05a9a318fc35f18cefa7e5059cfafc4362b737fa14e467441f3 not found: ID does not exist" containerID="ffe0b104ba9ba05a9a318fc35f18cefa7e5059cfafc4362b737fa14e467441f3" Oct 30 19:13:31.670173 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.670071 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ffe0b104ba9ba05a9a318fc35f18cefa7e5059cfafc4362b737fa14e467441f3"} err="failed to get container status \"ffe0b104ba9ba05a9a318fc35f18cefa7e5059cfafc4362b737fa14e467441f3\": rpc error: code = NotFound desc = could not find container \"ffe0b104ba9ba05a9a318fc35f18cefa7e5059cfafc4362b737fa14e467441f3\": container with ID starting with ffe0b104ba9ba05a9a318fc35f18cefa7e5059cfafc4362b737fa14e467441f3 not found: ID does not exist" Oct 30 19:13:31.794003 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.793960 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:31.812356 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.812322 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr"] Oct 30 19:13:31.861888 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:31.861820 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-m4lqr"] Oct 30 19:13:33.036597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:33.031921 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:33.651511 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:33.651480 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bcb8e536-b0a5-4517-8c98-a39793531709" path="/var/lib/kubelet/pods/bcb8e536-b0a5-4517-8c98-a39793531709/volumes" Oct 30 19:13:35.214696 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.214664 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:35.405619 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.403662 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:35.603671 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.595328 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs"] Oct 30 19:13:35.603671 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.598064 5894 cpu_manager.go:414] "containerMap: removing container" podUID="bcb8e536-b0a5-4517-8c98-a39793531709" containerName="istio-proxy" Oct 30 19:13:35.603671 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.598091 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bcb8e536-b0a5-4517-8c98-a39793531709" containerName="istio-proxy" Oct 30 19:13:35.603671 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.598266 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="bcb8e536-b0a5-4517-8c98-a39793531709" containerName="istio-proxy" Oct 30 19:13:35.607680 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.605021 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" Oct 30 19:13:35.648289 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.648258 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr2c4\" (UniqueName: \"kubernetes.io/projected/eb9bc7c3-4527-4569-b896-b302d1bc5afc-kube-api-access-cr2c4\") pod \"payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs\" (UID: \"eb9bc7c3-4527-4569-b896-b302d1bc5afc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" Oct 30 19:13:35.648450 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.648433 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb9bc7c3-4527-4569-b896-b302d1bc5afc-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs\" (UID: \"eb9bc7c3-4527-4569-b896-b302d1bc5afc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" Oct 30 19:13:35.654176 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.654152 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs"] Oct 30 19:13:35.729664 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.708268 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-e0fc1f\"" Oct 30 19:13:35.729664 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.708599 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-e0fc1f-dockercfg-fjkjg\"" Oct 30 19:13:35.752607 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.752125 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb9bc7c3-4527-4569-b896-b302d1bc5afc-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs\" (UID: \"eb9bc7c3-4527-4569-b896-b302d1bc5afc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" Oct 30 19:13:35.752607 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.752392 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr2c4\" (UniqueName: \"kubernetes.io/projected/eb9bc7c3-4527-4569-b896-b302d1bc5afc-kube-api-access-cr2c4\") pod \"payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs\" (UID: \"eb9bc7c3-4527-4569-b896-b302d1bc5afc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" Oct 30 19:13:35.761608 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.758966 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb9bc7c3-4527-4569-b896-b302d1bc5afc-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs\" (UID: \"eb9bc7c3-4527-4569-b896-b302d1bc5afc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" Oct 30 19:13:35.876689 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.861695 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct"] Oct 30 19:13:35.898616 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.887436 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr2c4\" (UniqueName: \"kubernetes.io/projected/eb9bc7c3-4527-4569-b896-b302d1bc5afc-kube-api-access-cr2c4\") pod \"payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs\" (UID: \"eb9bc7c3-4527-4569-b896-b302d1bc5afc\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" Oct 30 19:13:35.898616 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.896501 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" Oct 30 19:13:35.931004 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.930968 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" Oct 30 19:13:35.950485 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:35.950450 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct"] Oct 30 19:13:36.062116 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:36.062084 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b395010-277d-42b3-b191-0e24f69156c1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct\" (UID: \"1b395010-277d-42b3-b191-0e24f69156c1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" Oct 30 19:13:36.062275 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:36.062152 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxth6\" (UniqueName: \"kubernetes.io/projected/1b395010-277d-42b3-b191-0e24f69156c1-kube-api-access-kxth6\") pod \"payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct\" (UID: \"1b395010-277d-42b3-b191-0e24f69156c1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" Oct 30 19:13:36.163249 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:36.163203 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b395010-277d-42b3-b191-0e24f69156c1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct\" (UID: \"1b395010-277d-42b3-b191-0e24f69156c1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" Oct 30 19:13:36.163440 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:36.163284 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxth6\" (UniqueName: \"kubernetes.io/projected/1b395010-277d-42b3-b191-0e24f69156c1-kube-api-access-kxth6\") pod \"payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct\" (UID: \"1b395010-277d-42b3-b191-0e24f69156c1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" Oct 30 19:13:36.163941 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:36.163916 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b395010-277d-42b3-b191-0e24f69156c1-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct\" (UID: \"1b395010-277d-42b3-b191-0e24f69156c1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" Oct 30 19:13:36.220975 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:36.220941 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxth6\" (UniqueName: \"kubernetes.io/projected/1b395010-277d-42b3-b191-0e24f69156c1-kube-api-access-kxth6\") pod \"payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct\" (UID: \"1b395010-277d-42b3-b191-0e24f69156c1\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" Oct 30 19:13:36.248530 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:36.248503 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" Oct 30 19:13:36.345842 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:36.345813 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs"] Oct 30 19:13:36.347952 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:13:36.347923 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb9bc7c3_4527_4569_b896_b302d1bc5afc.slice/crio-6e93e46a266f14a6d220a3a0db4a0ebebc2bca8751b0e02587e1b78b4396c3ca WatchSource:0}: Error finding container 6e93e46a266f14a6d220a3a0db4a0ebebc2bca8751b0e02587e1b78b4396c3ca: Status 404 returned error can't find the container with id 6e93e46a266f14a6d220a3a0db4a0ebebc2bca8751b0e02587e1b78b4396c3ca Oct 30 19:13:36.630534 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:36.630508 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct"] Oct 30 19:13:36.631858 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:13:36.631834 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b395010_277d_42b3_b191_0e24f69156c1.slice/crio-8b2b2d2d18b350fe5e877eb58bd17ca9bbce292b126c945f1d8348553668afbc WatchSource:0}: Error finding container 8b2b2d2d18b350fe5e877eb58bd17ca9bbce292b126c945f1d8348553668afbc: Status 404 returned error can't find the container with id 8b2b2d2d18b350fe5e877eb58bd17ca9bbce292b126c945f1d8348553668afbc Oct 30 19:13:36.682695 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:36.682663 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" event={"ID":"1b395010-277d-42b3-b191-0e24f69156c1","Type":"ContainerStarted","Data":"8b2b2d2d18b350fe5e877eb58bd17ca9bbce292b126c945f1d8348553668afbc"} Oct 30 19:13:36.684613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:36.684558 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" event={"ID":"eb9bc7c3-4527-4569-b896-b302d1bc5afc","Type":"ContainerStarted","Data":"898b42ca95974646e1b62dbe90c030f7d79e238ab17ddffb3ccf42171e6f6cf5"} Oct 30 19:13:36.684711 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:36.684637 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" event={"ID":"eb9bc7c3-4527-4569-b896-b302d1bc5afc","Type":"ContainerStarted","Data":"6e93e46a266f14a6d220a3a0db4a0ebebc2bca8751b0e02587e1b78b4396c3ca"} Oct 30 19:13:36.684711 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:36.684704 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" Oct 30 19:13:36.797945 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:36.797859 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" podStartSLOduration=1.797841756 podStartE2EDuration="1.797841756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:13:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:13:36.797406585 +0000 UTC m=+2087.287534415" watchObservedRunningTime="2026-10-30 19:13:36.797841756 +0000 UTC m=+2087.287969576" Oct 30 19:13:37.331985 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:37.331953 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:37.505671 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:37.505626 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:37.693196 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:37.693160 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" event={"ID":"1b395010-277d-42b3-b191-0e24f69156c1","Type":"ContainerStarted","Data":"aa8be6fce6695580a0d73c25241124a922c7b7bb4b70f61e9796104c2b4b4d95"} Oct 30 19:13:37.693387 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:37.693229 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" Oct 30 19:13:37.809652 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:37.809564 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" podStartSLOduration=2.80954974 podStartE2EDuration="2.80954974s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:13:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:13:37.807869284 +0000 UTC m=+2088.297997104" watchObservedRunningTime="2026-10-30 19:13:37.80954974 +0000 UTC m=+2088.299677560" Oct 30 19:13:38.265207 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:38.265166 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:43.580024 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:43.579987 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:46.216783 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:46.209245 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:46.399036 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:46.398962 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:47.700983 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:47.700952 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" Oct 30 19:13:48.391853 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:48.391816 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4"] Oct 30 19:13:48.680030 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:48.679961 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:48.786595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:48.786475 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" Oct 30 19:13:48.869972 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:48.869928 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:49.069646 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:49.069541 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:50.201282 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:50.201254 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 19:13:50.219933 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:13:50.219904 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:13:50.232763 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:50.232737 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 19:13:53.392354 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:53.392293 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4" podUID="e56af8be-26ce-45af-963b-43797d0c0881" containerName="payload-pre-processing" containerID="cri-o://05bd3f4f2f2f9bdad61809b0ed6cd26b39e13ad21188a7ee7c02a0226e200b97" gracePeriod=25 Oct 30 19:13:53.704246 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:53.704214 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4" Oct 30 19:13:53.855745 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:53.855709 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e56af8be-26ce-45af-963b-43797d0c0881-kube-api-access-h8z5b\" (UniqueName: \"kubernetes.io/projected/e56af8be-26ce-45af-963b-43797d0c0881-kube-api-access-h8z5b\") pod \"e56af8be-26ce-45af-963b-43797d0c0881\" (UID: \"e56af8be-26ce-45af-963b-43797d0c0881\") " Oct 30 19:13:53.855931 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:53.855840 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e56af8be-26ce-45af-963b-43797d0c0881-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e56af8be-26ce-45af-963b-43797d0c0881-plugins-config-volume\") pod \"e56af8be-26ce-45af-963b-43797d0c0881\" (UID: \"e56af8be-26ce-45af-963b-43797d0c0881\") " Oct 30 19:13:53.856324 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:53.856296 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e56af8be-26ce-45af-963b-43797d0c0881-plugins-config-volume" pod "e56af8be-26ce-45af-963b-43797d0c0881" (UID: "e56af8be-26ce-45af-963b-43797d0c0881"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:13:53.856486 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:53.856460 5894 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e56af8be-26ce-45af-963b-43797d0c0881-plugins-config-volume\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:53.858103 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:53.858079 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e56af8be-26ce-45af-963b-43797d0c0881-kube-api-access-h8z5b" pod "e56af8be-26ce-45af-963b-43797d0c0881" (UID: "e56af8be-26ce-45af-963b-43797d0c0881"). InnerVolumeSpecName "kube-api-access-h8z5b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:13:53.903561 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:53.903535 5894 generic.go:356] "Generic (PLEG): container finished" podID="e56af8be-26ce-45af-963b-43797d0c0881" containerID="05bd3f4f2f2f9bdad61809b0ed6cd26b39e13ad21188a7ee7c02a0226e200b97" exitCode=0 Oct 30 19:13:53.903701 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:53.903616 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4" Oct 30 19:13:53.903701 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:53.903629 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4" event={"ID":"e56af8be-26ce-45af-963b-43797d0c0881","Type":"ContainerDied","Data":"05bd3f4f2f2f9bdad61809b0ed6cd26b39e13ad21188a7ee7c02a0226e200b97"} Oct 30 19:13:53.903701 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:53.903696 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4" event={"ID":"e56af8be-26ce-45af-963b-43797d0c0881","Type":"ContainerDied","Data":"16325e3a63a4e155cac32ef5362b3c30f38cee4fdf1e3a373a07c07abf3aa249"} Oct 30 19:13:53.903876 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:53.903726 5894 scope.go:122] "RemoveContainer" containerID="05bd3f4f2f2f9bdad61809b0ed6cd26b39e13ad21188a7ee7c02a0226e200b97" Oct 30 19:13:53.915053 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:53.915034 5894 scope.go:122] "RemoveContainer" containerID="05bd3f4f2f2f9bdad61809b0ed6cd26b39e13ad21188a7ee7c02a0226e200b97" Oct 30 19:13:53.915338 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:13:53.915315 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"05bd3f4f2f2f9bdad61809b0ed6cd26b39e13ad21188a7ee7c02a0226e200b97\": container with ID starting with 05bd3f4f2f2f9bdad61809b0ed6cd26b39e13ad21188a7ee7c02a0226e200b97 not found: ID does not exist" containerID="05bd3f4f2f2f9bdad61809b0ed6cd26b39e13ad21188a7ee7c02a0226e200b97" Oct 30 19:13:53.915403 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:53.915351 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"05bd3f4f2f2f9bdad61809b0ed6cd26b39e13ad21188a7ee7c02a0226e200b97"} err="failed to get container status \"05bd3f4f2f2f9bdad61809b0ed6cd26b39e13ad21188a7ee7c02a0226e200b97\": rpc error: code = NotFound desc = could not find container \"05bd3f4f2f2f9bdad61809b0ed6cd26b39e13ad21188a7ee7c02a0226e200b97\": container with ID starting with 05bd3f4f2f2f9bdad61809b0ed6cd26b39e13ad21188a7ee7c02a0226e200b97 not found: ID does not exist" Oct 30 19:13:53.957488 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:53.957463 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h8z5b\" (UniqueName: \"kubernetes.io/projected/e56af8be-26ce-45af-963b-43797d0c0881-kube-api-access-h8z5b\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:13:54.069165 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:54.069126 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4"] Oct 30 19:13:54.072602 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:54.072564 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-4ebeba-6b95467cc9-xs2n4"] Oct 30 19:13:54.089822 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:54.089797 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:13:55.659492 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:55.659455 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e56af8be-26ce-45af-963b-43797d0c0881" path="/var/lib/kubelet/pods/e56af8be-26ce-45af-963b-43797d0c0881/volumes" Oct 30 19:13:59.097949 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:13:59.097911 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:03.618954 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:03.618920 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:05.290618 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:05.290567 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:05.507551 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:05.507476 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:05.681747 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:05.681714 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:06.239958 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:06.239922 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:10.709271 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:10.709217 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:12.121291 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:12.121263 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:12.349316 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:12.349285 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:12.536610 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:12.536555 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:14.055830 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:14.055795 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:16.093521 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:16.093482 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:19.952385 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:19.952336 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x"] Oct 30 19:14:19.952917 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:19.952568 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" podUID="f6db387c-a643-49dd-9ef8-159fc42071d6" containerName="istio-proxy" containerID="cri-o://a07df988cb2c03d10d07da80b15d393016f2fd3d8fead4ebd512ec4773d1bb84" gracePeriod=30 Oct 30 19:14:20.120407 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:14:20.120370 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:14:20.683843 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:20.683804 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:20.870597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:20.865423 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:21.044537 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:21.044456 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:23.978946 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:23.978909 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:25.246937 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.246914 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:14:25.260744 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.260714 5894 generic.go:356] "Generic (PLEG): container finished" podID="f6db387c-a643-49dd-9ef8-159fc42071d6" containerID="a07df988cb2c03d10d07da80b15d393016f2fd3d8fead4ebd512ec4773d1bb84" exitCode=0 Oct 30 19:14:25.260871 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.260759 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" Oct 30 19:14:25.260871 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.260793 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" event={"ID":"f6db387c-a643-49dd-9ef8-159fc42071d6","Type":"ContainerDied","Data":"a07df988cb2c03d10d07da80b15d393016f2fd3d8fead4ebd512ec4773d1bb84"} Oct 30 19:14:25.260871 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.260846 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x" event={"ID":"f6db387c-a643-49dd-9ef8-159fc42071d6","Type":"ContainerDied","Data":"6a24c36f38ff454d11af0a92c94f80ad95a51c4576afe346fd6d76398d8ac1f8"} Oct 30 19:14:25.260977 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.260871 5894 scope.go:122] "RemoveContainer" containerID="a07df988cb2c03d10d07da80b15d393016f2fd3d8fead4ebd512ec4773d1bb84" Oct 30 19:14:25.271202 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.271185 5894 scope.go:122] "RemoveContainer" containerID="a07df988cb2c03d10d07da80b15d393016f2fd3d8fead4ebd512ec4773d1bb84" Oct 30 19:14:25.271526 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:14:25.271505 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a07df988cb2c03d10d07da80b15d393016f2fd3d8fead4ebd512ec4773d1bb84\": container with ID starting with a07df988cb2c03d10d07da80b15d393016f2fd3d8fead4ebd512ec4773d1bb84 not found: ID does not exist" containerID="a07df988cb2c03d10d07da80b15d393016f2fd3d8fead4ebd512ec4773d1bb84" Oct 30 19:14:25.271590 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.271535 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a07df988cb2c03d10d07da80b15d393016f2fd3d8fead4ebd512ec4773d1bb84"} err="failed to get container status \"a07df988cb2c03d10d07da80b15d393016f2fd3d8fead4ebd512ec4773d1bb84\": rpc error: code = NotFound desc = could not find container \"a07df988cb2c03d10d07da80b15d393016f2fd3d8fead4ebd512ec4773d1bb84\": container with ID starting with a07df988cb2c03d10d07da80b15d393016f2fd3d8fead4ebd512ec4773d1bb84 not found: ID does not exist" Oct 30 19:14:25.370611 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.370563 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-data\") pod \"f6db387c-a643-49dd-9ef8-159fc42071d6\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " Oct 30 19:14:25.370765 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.370622 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-credential-socket\") pod \"f6db387c-a643-49dd-9ef8-159fc42071d6\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " Oct 30 19:14:25.370765 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.370656 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-envoy\") pod \"f6db387c-a643-49dd-9ef8-159fc42071d6\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " Oct 30 19:14:25.370765 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.370745 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-token\" (UniqueName: \"kubernetes.io/projected/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-token\") pod \"f6db387c-a643-49dd-9ef8-159fc42071d6\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " Oct 30 19:14:25.370896 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.370800 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f6db387c-a643-49dd-9ef8-159fc42071d6-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6db387c-a643-49dd-9ef8-159fc42071d6-istiod-ca-cert\") pod \"f6db387c-a643-49dd-9ef8-159fc42071d6\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " Oct 30 19:14:25.370896 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.370826 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f6db387c-a643-49dd-9ef8-159fc42071d6-kube-api-access-l92pj\" (UniqueName: \"kubernetes.io/projected/f6db387c-a643-49dd-9ef8-159fc42071d6-kube-api-access-l92pj\") pod \"f6db387c-a643-49dd-9ef8-159fc42071d6\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " Oct 30 19:14:25.370896 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.370882 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-data" pod "f6db387c-a643-49dd-9ef8-159fc42071d6" (UID: "f6db387c-a643-49dd-9ef8-159fc42071d6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:14:25.371062 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.370902 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-credential-socket" pod "f6db387c-a643-49dd-9ef8-159fc42071d6" (UID: "f6db387c-a643-49dd-9ef8-159fc42071d6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:14:25.371062 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.370959 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-workload-certs\") pod \"f6db387c-a643-49dd-9ef8-159fc42071d6\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " Oct 30 19:14:25.371062 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.371008 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-workload-socket\") pod \"f6db387c-a643-49dd-9ef8-159fc42071d6\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " Oct 30 19:14:25.371221 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.371084 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-podinfo\") pod \"f6db387c-a643-49dd-9ef8-159fc42071d6\" (UID: \"f6db387c-a643-49dd-9ef8-159fc42071d6\") " Oct 30 19:14:25.371278 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.371239 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-workload-certs" pod "f6db387c-a643-49dd-9ef8-159fc42071d6" (UID: "f6db387c-a643-49dd-9ef8-159fc42071d6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:14:25.371278 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.371254 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-workload-socket" pod "f6db387c-a643-49dd-9ef8-159fc42071d6" (UID: "f6db387c-a643-49dd-9ef8-159fc42071d6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:14:25.371384 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.371289 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f6db387c-a643-49dd-9ef8-159fc42071d6-istiod-ca-cert" pod "f6db387c-a643-49dd-9ef8-159fc42071d6" (UID: "f6db387c-a643-49dd-9ef8-159fc42071d6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:14:25.371682 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.371661 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:25.371781 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.371687 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:25.371781 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.371702 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f6db387c-a643-49dd-9ef8-159fc42071d6-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:25.371781 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.371716 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:25.371781 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.371729 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:25.373154 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.373130 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-podinfo" pod "f6db387c-a643-49dd-9ef8-159fc42071d6" (UID: "f6db387c-a643-49dd-9ef8-159fc42071d6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:14:25.373154 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.373141 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6db387c-a643-49dd-9ef8-159fc42071d6-kube-api-access-l92pj" pod "f6db387c-a643-49dd-9ef8-159fc42071d6" (UID: "f6db387c-a643-49dd-9ef8-159fc42071d6"). InnerVolumeSpecName "kube-api-access-l92pj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:14:25.373361 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.373341 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-envoy" pod "f6db387c-a643-49dd-9ef8-159fc42071d6" (UID: "f6db387c-a643-49dd-9ef8-159fc42071d6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:14:25.373426 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.373347 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-token" pod "f6db387c-a643-49dd-9ef8-159fc42071d6" (UID: "f6db387c-a643-49dd-9ef8-159fc42071d6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:14:25.473992 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.473891 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:25.474196 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.474179 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:25.474291 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.474277 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l92pj\" (UniqueName: \"kubernetes.io/projected/f6db387c-a643-49dd-9ef8-159fc42071d6-kube-api-access-l92pj\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:25.474378 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.474367 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f6db387c-a643-49dd-9ef8-159fc42071d6-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:25.713693 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.713649 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x"] Oct 30 19:14:25.728361 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.717751 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:25.770203 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.770162 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-drj7x"] Oct 30 19:14:25.872464 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.872430 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps"] Oct 30 19:14:25.872679 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:25.872656 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" podUID="62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" containerName="istio-proxy" containerID="cri-o://582f6fa875c3e25dfca56050790e94b17539fa081567e399f1177034a8170a8b" gracePeriod=30 Oct 30 19:14:27.651481 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:27.651450 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f6db387c-a643-49dd-9ef8-159fc42071d6" path="/var/lib/kubelet/pods/f6db387c-a643-49dd-9ef8-159fc42071d6/volumes" Oct 30 19:14:27.752678 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:27.752647 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:14:27.752877 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:27.752725 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" podUID="62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:14:28.189020 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:28.188985 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs"] Oct 30 19:14:28.591258 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:28.591172 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:29.762747 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:29.735997 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97"] Oct 30 19:14:29.762747 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:29.736273 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" podUID="63ba4b70-66a0-4808-9ff1-f413a6cd6a35" containerName="maas-api" containerID="cri-o://a2bb2a8d7cdc065a5ed8d2c4ce88f6e8b715922021a34ba7d00e37d03e811cc5" gracePeriod=30 Oct 30 19:14:30.297892 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.297859 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" Oct 30 19:14:30.298348 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.298327 5894 generic.go:356] "Generic (PLEG): container finished" podID="63ba4b70-66a0-4808-9ff1-f413a6cd6a35" containerID="a2bb2a8d7cdc065a5ed8d2c4ce88f6e8b715922021a34ba7d00e37d03e811cc5" exitCode=0 Oct 30 19:14:30.298447 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.298406 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" event={"ID":"63ba4b70-66a0-4808-9ff1-f413a6cd6a35","Type":"ContainerDied","Data":"a2bb2a8d7cdc065a5ed8d2c4ce88f6e8b715922021a34ba7d00e37d03e811cc5"} Oct 30 19:14:30.298507 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.298469 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" event={"ID":"63ba4b70-66a0-4808-9ff1-f413a6cd6a35","Type":"ContainerDied","Data":"831a131c3364e4ea6149ccb2ae358a8d4b6dd07c150b0b16bfef2d698d6c900a"} Oct 30 19:14:30.298507 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.298490 5894 scope.go:122] "RemoveContainer" containerID="a2bb2a8d7cdc065a5ed8d2c4ce88f6e8b715922021a34ba7d00e37d03e811cc5" Oct 30 19:14:30.309762 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.309740 5894 scope.go:122] "RemoveContainer" containerID="a2bb2a8d7cdc065a5ed8d2c4ce88f6e8b715922021a34ba7d00e37d03e811cc5" Oct 30 19:14:30.310067 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:14:30.310040 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2bb2a8d7cdc065a5ed8d2c4ce88f6e8b715922021a34ba7d00e37d03e811cc5\": container with ID starting with a2bb2a8d7cdc065a5ed8d2c4ce88f6e8b715922021a34ba7d00e37d03e811cc5 not found: ID does not exist" containerID="a2bb2a8d7cdc065a5ed8d2c4ce88f6e8b715922021a34ba7d00e37d03e811cc5" Oct 30 19:14:30.310158 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.310080 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2bb2a8d7cdc065a5ed8d2c4ce88f6e8b715922021a34ba7d00e37d03e811cc5"} err="failed to get container status \"a2bb2a8d7cdc065a5ed8d2c4ce88f6e8b715922021a34ba7d00e37d03e811cc5\": rpc error: code = NotFound desc = could not find container \"a2bb2a8d7cdc065a5ed8d2c4ce88f6e8b715922021a34ba7d00e37d03e811cc5\": container with ID starting with a2bb2a8d7cdc065a5ed8d2c4ce88f6e8b715922021a34ba7d00e37d03e811cc5 not found: ID does not exist" Oct 30 19:14:30.376248 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.376212 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-openshift-service-ca\") pod \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\" (UID: \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\") " Oct 30 19:14:30.376395 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.376294 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-kube-api-access-mbbmd\" (UniqueName: \"kubernetes.io/projected/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-kube-api-access-mbbmd\") pod \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\" (UID: \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\") " Oct 30 19:14:30.376437 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.376399 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-maas-api-tls\") pod \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\" (UID: \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\") " Oct 30 19:14:30.376499 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.376483 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-metrics-certs\" (UniqueName: \"kubernetes.io/secret/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-metrics-certs\") pod \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\" (UID: \"63ba4b70-66a0-4808-9ff1-f413a6cd6a35\") " Oct 30 19:14:30.376640 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.376566 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-openshift-service-ca" pod "63ba4b70-66a0-4808-9ff1-f413a6cd6a35" (UID: "63ba4b70-66a0-4808-9ff1-f413a6cd6a35"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:14:30.377081 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.377060 5894 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-openshift-service-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:30.378534 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.378509 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-maas-api-tls" pod "63ba4b70-66a0-4808-9ff1-f413a6cd6a35" (UID: "63ba4b70-66a0-4808-9ff1-f413a6cd6a35"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:14:30.378678 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.378613 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-kube-api-access-mbbmd" pod "63ba4b70-66a0-4808-9ff1-f413a6cd6a35" (UID: "63ba4b70-66a0-4808-9ff1-f413a6cd6a35"). InnerVolumeSpecName "kube-api-access-mbbmd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:14:30.378727 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.378703 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-metrics-certs" pod "63ba4b70-66a0-4808-9ff1-f413a6cd6a35" (UID: "63ba4b70-66a0-4808-9ff1-f413a6cd6a35"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:14:30.478901 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.478860 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-maas-api-tls\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:30.478901 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.478900 5894 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-metrics-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:30.479154 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.478947 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mbbmd\" (UniqueName: \"kubernetes.io/projected/63ba4b70-66a0-4808-9ff1-f413a6cd6a35-kube-api-access-mbbmd\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:30.890665 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:30.890629 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:31.154830 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.154806 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 19:14:31.188815 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.188777 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-credential-socket\") pod \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " Oct 30 19:14:31.188959 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.188839 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-envoy\") pod \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " Oct 30 19:14:31.188959 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.188860 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-token\" (UniqueName: \"kubernetes.io/projected/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-token\") pod \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " Oct 30 19:14:31.189083 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.189045 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-credential-socket" pod "62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" (UID: "62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:14:31.189083 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.189067 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-workload-socket\") pod \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " Oct 30 19:14:31.189195 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.189100 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-kube-api-access-9vmsm\" (UniqueName: \"kubernetes.io/projected/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-kube-api-access-9vmsm\") pod \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " Oct 30 19:14:31.189195 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.189137 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istiod-ca-cert\") pod \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " Oct 30 19:14:31.189303 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.189201 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-workload-certs\") pod \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " Oct 30 19:14:31.189367 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.189333 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-workload-socket" pod "62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" (UID: "62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:14:31.189367 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.189352 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-podinfo\") pod \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " Oct 30 19:14:31.189464 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.189432 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-data\") pod \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\" (UID: \"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66\") " Oct 30 19:14:31.189550 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.189508 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-workload-certs" pod "62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" (UID: "62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:14:31.189550 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.189535 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istiod-ca-cert" pod "62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" (UID: "62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:14:31.189724 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.189702 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-data" pod "62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" (UID: "62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:14:31.190010 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.189990 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:31.190010 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.190011 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:31.190148 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.190020 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:31.190148 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.190028 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:31.190148 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.190037 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:31.191217 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.191193 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-envoy" pod "62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" (UID: "62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:14:31.191329 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.191279 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-kube-api-access-9vmsm" pod "62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" (UID: "62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66"). InnerVolumeSpecName "kube-api-access-9vmsm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:14:31.191676 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.191655 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-podinfo" pod "62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" (UID: "62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:14:31.191732 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.191691 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-token" pod "62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" (UID: "62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:14:31.291038 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.290998 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:31.291038 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.291038 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:31.291038 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.291047 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vmsm\" (UniqueName: \"kubernetes.io/projected/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-kube-api-access-9vmsm\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:31.291238 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.291057 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:31.306695 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.306667 5894 generic.go:356] "Generic (PLEG): container finished" podID="62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" containerID="582f6fa875c3e25dfca56050790e94b17539fa081567e399f1177034a8170a8b" exitCode=0 Oct 30 19:14:31.306799 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.306719 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" Oct 30 19:14:31.306799 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.306751 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" event={"ID":"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66","Type":"ContainerDied","Data":"582f6fa875c3e25dfca56050790e94b17539fa081567e399f1177034a8170a8b"} Oct 30 19:14:31.306924 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.306806 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps" event={"ID":"62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66","Type":"ContainerDied","Data":"f3fedc5bccf2c949134bdc95b710884918527e73d5b63d7a7e4828ee409bf826"} Oct 30 19:14:31.306924 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.306836 5894 scope.go:122] "RemoveContainer" containerID="582f6fa875c3e25dfca56050790e94b17539fa081567e399f1177034a8170a8b" Oct 30 19:14:31.308037 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.308019 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97" Oct 30 19:14:31.318255 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.318183 5894 scope.go:122] "RemoveContainer" containerID="582f6fa875c3e25dfca56050790e94b17539fa081567e399f1177034a8170a8b" Oct 30 19:14:31.318593 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:14:31.318554 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"582f6fa875c3e25dfca56050790e94b17539fa081567e399f1177034a8170a8b\": container with ID starting with 582f6fa875c3e25dfca56050790e94b17539fa081567e399f1177034a8170a8b not found: ID does not exist" containerID="582f6fa875c3e25dfca56050790e94b17539fa081567e399f1177034a8170a8b" Oct 30 19:14:31.318676 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.318600 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"582f6fa875c3e25dfca56050790e94b17539fa081567e399f1177034a8170a8b"} err="failed to get container status \"582f6fa875c3e25dfca56050790e94b17539fa081567e399f1177034a8170a8b\": rpc error: code = NotFound desc = could not find container \"582f6fa875c3e25dfca56050790e94b17539fa081567e399f1177034a8170a8b\": container with ID starting with 582f6fa875c3e25dfca56050790e94b17539fa081567e399f1177034a8170a8b not found: ID does not exist" Oct 30 19:14:31.436076 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.435985 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97"] Oct 30 19:14:31.439782 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.439756 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-w1-7d059d-5497d74969-4cw97"] Oct 30 19:14:31.548908 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.548868 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps"] Oct 30 19:14:31.552149 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.552121 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-5vrps"] Oct 30 19:14:31.579617 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.578792 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:31.653283 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.653250 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" path="/var/lib/kubelet/pods/62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66/volumes" Oct 30 19:14:31.653800 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:31.653785 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="63ba4b70-66a0-4808-9ff1-f413a6cd6a35" path="/var/lib/kubelet/pods/63ba4b70-66a0-4808-9ff1-f413a6cd6a35/volumes" Oct 30 19:14:33.281647 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:33.281609 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs" podUID="fc0db36f-b2b2-4add-987b-c0425935d5ae" containerName="payload-processing" containerID="cri-o://2a96f01078b383102416474b188fb923f01f2b34035c5d0bff1ffaca020ce25d" gracePeriod=25 Oct 30 19:14:33.578748 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:33.578722 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs" Oct 30 19:14:33.630391 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:33.630362 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fc0db36f-b2b2-4add-987b-c0425935d5ae-kube-api-access-kzl4f\" (UniqueName: \"kubernetes.io/projected/fc0db36f-b2b2-4add-987b-c0425935d5ae-kube-api-access-kzl4f\") pod \"fc0db36f-b2b2-4add-987b-c0425935d5ae\" (UID: \"fc0db36f-b2b2-4add-987b-c0425935d5ae\") " Oct 30 19:14:33.630549 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:33.630534 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/fc0db36f-b2b2-4add-987b-c0425935d5ae-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc0db36f-b2b2-4add-987b-c0425935d5ae-plugins-config-volume\") pod \"fc0db36f-b2b2-4add-987b-c0425935d5ae\" (UID: \"fc0db36f-b2b2-4add-987b-c0425935d5ae\") " Oct 30 19:14:33.630981 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:33.630947 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fc0db36f-b2b2-4add-987b-c0425935d5ae-plugins-config-volume" pod "fc0db36f-b2b2-4add-987b-c0425935d5ae" (UID: "fc0db36f-b2b2-4add-987b-c0425935d5ae"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:14:33.631129 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:33.631103 5894 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fc0db36f-b2b2-4add-987b-c0425935d5ae-plugins-config-volume\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:33.632431 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:33.632408 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc0db36f-b2b2-4add-987b-c0425935d5ae-kube-api-access-kzl4f" pod "fc0db36f-b2b2-4add-987b-c0425935d5ae" (UID: "fc0db36f-b2b2-4add-987b-c0425935d5ae"). InnerVolumeSpecName "kube-api-access-kzl4f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:14:33.732211 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:33.732180 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kzl4f\" (UniqueName: \"kubernetes.io/projected/fc0db36f-b2b2-4add-987b-c0425935d5ae-kube-api-access-kzl4f\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:14:34.330027 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:34.329985 5894 generic.go:356] "Generic (PLEG): container finished" podID="fc0db36f-b2b2-4add-987b-c0425935d5ae" containerID="2a96f01078b383102416474b188fb923f01f2b34035c5d0bff1ffaca020ce25d" exitCode=0 Oct 30 19:14:34.330473 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:34.330052 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs" Oct 30 19:14:34.330473 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:34.330073 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs" event={"ID":"fc0db36f-b2b2-4add-987b-c0425935d5ae","Type":"ContainerDied","Data":"2a96f01078b383102416474b188fb923f01f2b34035c5d0bff1ffaca020ce25d"} Oct 30 19:14:34.330473 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:34.330156 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs" event={"ID":"fc0db36f-b2b2-4add-987b-c0425935d5ae","Type":"ContainerDied","Data":"5b9ff0641ad04ae0fa18a9af5e02a36e5e5b9d48c2b380849e31557d3621dcc6"} Oct 30 19:14:34.330473 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:34.330180 5894 scope.go:122] "RemoveContainer" containerID="2a96f01078b383102416474b188fb923f01f2b34035c5d0bff1ffaca020ce25d" Oct 30 19:14:34.340658 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:34.340633 5894 scope.go:122] "RemoveContainer" containerID="2a96f01078b383102416474b188fb923f01f2b34035c5d0bff1ffaca020ce25d" Oct 30 19:14:34.340960 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:14:34.340937 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a96f01078b383102416474b188fb923f01f2b34035c5d0bff1ffaca020ce25d\": container with ID starting with 2a96f01078b383102416474b188fb923f01f2b34035c5d0bff1ffaca020ce25d not found: ID does not exist" containerID="2a96f01078b383102416474b188fb923f01f2b34035c5d0bff1ffaca020ce25d" Oct 30 19:14:34.341014 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:34.340970 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a96f01078b383102416474b188fb923f01f2b34035c5d0bff1ffaca020ce25d"} err="failed to get container status \"2a96f01078b383102416474b188fb923f01f2b34035c5d0bff1ffaca020ce25d\": rpc error: code = NotFound desc = could not find container \"2a96f01078b383102416474b188fb923f01f2b34035c5d0bff1ffaca020ce25d\": container with ID starting with 2a96f01078b383102416474b188fb923f01f2b34035c5d0bff1ffaca020ce25d not found: ID does not exist" Oct 30 19:14:34.493830 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:34.493798 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs"] Oct 30 19:14:34.543385 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:34.543357 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-7d059d-6fb6c8d789-dghfs"] Oct 30 19:14:35.651039 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:35.651007 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fc0db36f-b2b2-4add-987b-c0425935d5ae" path="/var/lib/kubelet/pods/fc0db36f-b2b2-4add-987b-c0425935d5ae/volumes" Oct 30 19:14:36.212512 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:36.212476 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:40.996535 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:40.996495 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:45.609229 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.609189 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv"] Oct 30 19:14:45.610182 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.610159 5894 cpu_manager.go:414] "containerMap: removing container" podUID="e56af8be-26ce-45af-963b-43797d0c0881" containerName="payload-pre-processing" Oct 30 19:14:45.610245 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.610186 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e56af8be-26ce-45af-963b-43797d0c0881" containerName="payload-pre-processing" Oct 30 19:14:45.610245 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.610205 5894 cpu_manager.go:414] "containerMap: removing container" podUID="62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" containerName="istio-proxy" Oct 30 19:14:45.610245 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.610214 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" containerName="istio-proxy" Oct 30 19:14:45.610245 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.610228 5894 cpu_manager.go:414] "containerMap: removing container" podUID="63ba4b70-66a0-4808-9ff1-f413a6cd6a35" containerName="maas-api" Oct 30 19:14:45.610245 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.610236 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="63ba4b70-66a0-4808-9ff1-f413a6cd6a35" containerName="maas-api" Oct 30 19:14:45.610400 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.610265 5894 cpu_manager.go:414] "containerMap: removing container" podUID="f6db387c-a643-49dd-9ef8-159fc42071d6" containerName="istio-proxy" Oct 30 19:14:45.610400 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.610272 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f6db387c-a643-49dd-9ef8-159fc42071d6" containerName="istio-proxy" Oct 30 19:14:45.610400 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.610294 5894 cpu_manager.go:414] "containerMap: removing container" podUID="fc0db36f-b2b2-4add-987b-c0425935d5ae" containerName="payload-processing" Oct 30 19:14:45.610400 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.610302 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fc0db36f-b2b2-4add-987b-c0425935d5ae" containerName="payload-processing" Oct 30 19:14:45.610530 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.610498 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="e56af8be-26ce-45af-963b-43797d0c0881" containerName="payload-pre-processing" Oct 30 19:14:45.610530 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.610512 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="62f6a40a-acbe-4f4b-98c9-6a3ed3ecad66" containerName="istio-proxy" Oct 30 19:14:45.610530 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.610523 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="63ba4b70-66a0-4808-9ff1-f413a6cd6a35" containerName="maas-api" Oct 30 19:14:45.610646 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.610535 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="f6db387c-a643-49dd-9ef8-159fc42071d6" containerName="istio-proxy" Oct 30 19:14:45.610646 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.610545 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="fc0db36f-b2b2-4add-987b-c0425935d5ae" containerName="payload-processing" Oct 30 19:14:45.614672 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.614649 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" Oct 30 19:14:45.666666 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.666640 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-ae6a138d-serving-cert\"" Oct 30 19:14:45.702728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.702702 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03ef4756-9a48-4f73-9229-6e7cc936e052-openshift-service-ca\") pod \"maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv\" (UID: \"03ef4756-9a48-4f73-9229-6e7cc936e052\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" Oct 30 19:14:45.702866 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.702852 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03ef4756-9a48-4f73-9229-6e7cc936e052-metrics-certs\") pod \"maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv\" (UID: \"03ef4756-9a48-4f73-9229-6e7cc936e052\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" Oct 30 19:14:45.702913 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.702899 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bq59v\" (UniqueName: \"kubernetes.io/projected/03ef4756-9a48-4f73-9229-6e7cc936e052-kube-api-access-bq59v\") pod \"maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv\" (UID: \"03ef4756-9a48-4f73-9229-6e7cc936e052\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" Oct 30 19:14:45.703021 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.703005 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03ef4756-9a48-4f73-9229-6e7cc936e052-maas-api-tls\") pod \"maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv\" (UID: \"03ef4756-9a48-4f73-9229-6e7cc936e052\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" Oct 30 19:14:45.721094 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.721062 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv"] Oct 30 19:14:45.805494 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.805462 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03ef4756-9a48-4f73-9229-6e7cc936e052-maas-api-tls\") pod \"maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv\" (UID: \"03ef4756-9a48-4f73-9229-6e7cc936e052\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" Oct 30 19:14:45.805688 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.805630 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03ef4756-9a48-4f73-9229-6e7cc936e052-openshift-service-ca\") pod \"maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv\" (UID: \"03ef4756-9a48-4f73-9229-6e7cc936e052\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" Oct 30 19:14:45.805936 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:14:45.805913 5894 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-serving-cert: secret "maas-api-e2e-mt-ae6a138d-serving-cert" not found Oct 30 19:14:45.806446 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.806411 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03ef4756-9a48-4f73-9229-6e7cc936e052-openshift-service-ca\") pod \"maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv\" (UID: \"03ef4756-9a48-4f73-9229-6e7cc936e052\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" Oct 30 19:14:45.806699 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:14:45.806676 5894 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/03ef4756-9a48-4f73-9229-6e7cc936e052-maas-api-tls podName:03ef4756-9a48-4f73-9229-6e7cc936e052 nodeName:}" failed. No retries permitted until 2026-10-30 19:14:46.306652995 +0000 UTC m=+2156.796780815 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/03ef4756-9a48-4f73-9229-6e7cc936e052-maas-api-tls") pod "maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" (UID: "03ef4756-9a48-4f73-9229-6e7cc936e052") : secret "maas-api-e2e-mt-ae6a138d-serving-cert" not found Oct 30 19:14:45.806889 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.806869 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03ef4756-9a48-4f73-9229-6e7cc936e052-metrics-certs\") pod \"maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv\" (UID: \"03ef4756-9a48-4f73-9229-6e7cc936e052\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" Oct 30 19:14:45.806961 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.806935 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bq59v\" (UniqueName: \"kubernetes.io/projected/03ef4756-9a48-4f73-9229-6e7cc936e052-kube-api-access-bq59v\") pod \"maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv\" (UID: \"03ef4756-9a48-4f73-9229-6e7cc936e052\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" Oct 30 19:14:45.809721 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.809696 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03ef4756-9a48-4f73-9229-6e7cc936e052-metrics-certs\") pod \"maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv\" (UID: \"03ef4756-9a48-4f73-9229-6e7cc936e052\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" Oct 30 19:14:45.865302 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.865230 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bq59v\" (UniqueName: \"kubernetes.io/projected/03ef4756-9a48-4f73-9229-6e7cc936e052-kube-api-access-bq59v\") pod \"maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv\" (UID: \"03ef4756-9a48-4f73-9229-6e7cc936e052\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" Oct 30 19:14:45.925598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:45.917418 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:46.107989 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:46.107948 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:46.351292 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:46.351257 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03ef4756-9a48-4f73-9229-6e7cc936e052-maas-api-tls\") pod \"maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv\" (UID: \"03ef4756-9a48-4f73-9229-6e7cc936e052\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" Oct 30 19:14:46.353784 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:46.353763 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03ef4756-9a48-4f73-9229-6e7cc936e052-maas-api-tls\") pod \"maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv\" (UID: \"03ef4756-9a48-4f73-9229-6e7cc936e052\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" Oct 30 19:14:46.358556 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:46.358531 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:46.529447 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:46.529411 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" Oct 30 19:14:46.909780 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:46.909749 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv"] Oct 30 19:14:46.910838 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:14:46.910815 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03ef4756_9a48_4f73_9229_6e7cc936e052.slice/crio-2aecc585c96c963bf3f38124529136913f550a8ec36b11b5257303af9cbb1501 WatchSource:0}: Error finding container 2aecc585c96c963bf3f38124529136913f550a8ec36b11b5257303af9cbb1501: Status 404 returned error can't find the container with id 2aecc585c96c963bf3f38124529136913f550a8ec36b11b5257303af9cbb1501 Oct 30 19:14:46.912309 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:46.912287 5894 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Oct 30 19:14:47.420055 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:47.420022 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" event={"ID":"03ef4756-9a48-4f73-9229-6e7cc936e052","Type":"ContainerStarted","Data":"2aecc585c96c963bf3f38124529136913f550a8ec36b11b5257303af9cbb1501"} Oct 30 19:14:48.428007 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:48.427974 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" event={"ID":"03ef4756-9a48-4f73-9229-6e7cc936e052","Type":"ContainerStarted","Data":"7727b606e714ba69ec156d47f2dff87a0d9d2f4107f75ab15b595a4568597bf8"} Oct 30 19:14:48.428380 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:48.428096 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" Oct 30 19:14:48.545556 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:48.545419 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" podStartSLOduration=2.702880496 podStartE2EDuration="3.545405239s" totalImagesPullingTime="842.524743ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:14:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:14:48.543297353 +0000 UTC m=+2159.033425173" watchObservedRunningTime="2026-10-30 19:14:48.545405239 +0000 UTC m=+2159.035533085" Oct 30 19:14:50.129832 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:14:50.129793 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:14:51.239178 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:51.239135 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:52.548055 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:52.548019 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:52.739049 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:52.739013 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:54.439783 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:54.439753 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" Oct 30 19:14:55.351037 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:55.351003 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:56.959150 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:56.959109 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:57.174596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:57.166801 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:57.339736 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:57.339653 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:57.519152 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:57.519094 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:14:57.865637 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:14:57.865599 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:00.206119 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.206089 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s"] Oct 30 19:15:00.222432 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.222390 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph"] Oct 30 19:15:00.222635 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.222481 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" Oct 30 19:15:00.227169 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.227144 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" Oct 30 19:15:00.245243 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.245219 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn"] Oct 30 19:15:00.251667 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.251647 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" Oct 30 19:15:00.264162 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.264138 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d"] Oct 30 19:15:00.268098 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.268078 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" Oct 30 19:15:00.316227 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.316187 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s"] Oct 30 19:15:00.322028 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.321991 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph"] Oct 30 19:15:00.329373 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.329344 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nqq9m\"" Oct 30 19:15:00.347486 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.347457 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rzl7n\" (UniqueName: \"kubernetes.io/projected/416b873d-e218-44c4-8b1d-748d2329ae43-kube-api-access-rzl7n\") pod \"maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d\" (UID: \"416b873d-e218-44c4-8b1d-748d2329ae43\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" Oct 30 19:15:00.348122 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.347507 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6wjz\" (UniqueName: \"kubernetes.io/projected/3c1a3f52-3da7-417c-a4e1-a443204576d5-kube-api-access-d6wjz\") pod \"maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn\" (UID: \"3c1a3f52-3da7-417c-a4e1-a443204576d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" Oct 30 19:15:00.348122 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.347716 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nb45z\" (UniqueName: \"kubernetes.io/projected/3ff1c8e5-4048-4daa-88df-5287e0e456c6-kube-api-access-nb45z\") pod \"maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s\" (UID: \"3ff1c8e5-4048-4daa-88df-5287e0e456c6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" Oct 30 19:15:00.348122 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.347780 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/290f6845-8233-4640-80a5-d25c79c76339-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph\" (UID: \"290f6845-8233-4640-80a5-d25c79c76339\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" Oct 30 19:15:00.348122 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.347833 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/416b873d-e218-44c4-8b1d-748d2329ae43-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d\" (UID: \"416b873d-e218-44c4-8b1d-748d2329ae43\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" Oct 30 19:15:00.348122 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.347856 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x94z2\" (UniqueName: \"kubernetes.io/projected/290f6845-8233-4640-80a5-d25c79c76339-kube-api-access-x94z2\") pod \"maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph\" (UID: \"290f6845-8233-4640-80a5-d25c79c76339\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" Oct 30 19:15:00.348122 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.347889 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c1a3f52-3da7-417c-a4e1-a443204576d5-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn\" (UID: \"3c1a3f52-3da7-417c-a4e1-a443204576d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" Oct 30 19:15:00.348122 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.347922 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ff1c8e5-4048-4daa-88df-5287e0e456c6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s\" (UID: \"3ff1c8e5-4048-4daa-88df-5287e0e456c6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" Oct 30 19:15:00.364970 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.364940 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn"] Oct 30 19:15:00.368822 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.368797 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d"] Oct 30 19:15:00.453327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.453274 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/290f6845-8233-4640-80a5-d25c79c76339-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph\" (UID: \"290f6845-8233-4640-80a5-d25c79c76339\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" Oct 30 19:15:00.454500 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.454475 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/416b873d-e218-44c4-8b1d-748d2329ae43-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d\" (UID: \"416b873d-e218-44c4-8b1d-748d2329ae43\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" Oct 30 19:15:00.455528 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.455510 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x94z2\" (UniqueName: \"kubernetes.io/projected/290f6845-8233-4640-80a5-d25c79c76339-kube-api-access-x94z2\") pod \"maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph\" (UID: \"290f6845-8233-4640-80a5-d25c79c76339\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" Oct 30 19:15:00.456034 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.456012 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c1a3f52-3da7-417c-a4e1-a443204576d5-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn\" (UID: \"3c1a3f52-3da7-417c-a4e1-a443204576d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" Oct 30 19:15:00.456996 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.456912 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ff1c8e5-4048-4daa-88df-5287e0e456c6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s\" (UID: \"3ff1c8e5-4048-4daa-88df-5287e0e456c6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" Oct 30 19:15:00.458239 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.458206 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rzl7n\" (UniqueName: \"kubernetes.io/projected/416b873d-e218-44c4-8b1d-748d2329ae43-kube-api-access-rzl7n\") pod \"maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d\" (UID: \"416b873d-e218-44c4-8b1d-748d2329ae43\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" Oct 30 19:15:00.458782 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.458753 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6wjz\" (UniqueName: \"kubernetes.io/projected/3c1a3f52-3da7-417c-a4e1-a443204576d5-kube-api-access-d6wjz\") pod \"maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn\" (UID: \"3c1a3f52-3da7-417c-a4e1-a443204576d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" Oct 30 19:15:00.459322 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.459301 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nb45z\" (UniqueName: \"kubernetes.io/projected/3ff1c8e5-4048-4daa-88df-5287e0e456c6-kube-api-access-nb45z\") pod \"maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s\" (UID: \"3ff1c8e5-4048-4daa-88df-5287e0e456c6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" Oct 30 19:15:00.459823 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.458043 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ff1c8e5-4048-4daa-88df-5287e0e456c6-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s\" (UID: \"3ff1c8e5-4048-4daa-88df-5287e0e456c6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" Oct 30 19:15:00.459932 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.455436 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/416b873d-e218-44c4-8b1d-748d2329ae43-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d\" (UID: \"416b873d-e218-44c4-8b1d-748d2329ae43\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" Oct 30 19:15:00.459932 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.456829 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c1a3f52-3da7-417c-a4e1-a443204576d5-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn\" (UID: \"3c1a3f52-3da7-417c-a4e1-a443204576d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" Oct 30 19:15:00.459932 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.454329 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/290f6845-8233-4640-80a5-d25c79c76339-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph\" (UID: \"290f6845-8233-4640-80a5-d25c79c76339\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" Oct 30 19:15:00.531839 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.531801 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x94z2\" (UniqueName: \"kubernetes.io/projected/290f6845-8233-4640-80a5-d25c79c76339-kube-api-access-x94z2\") pod \"maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph\" (UID: \"290f6845-8233-4640-80a5-d25c79c76339\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" Oct 30 19:15:00.534217 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.534190 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6wjz\" (UniqueName: \"kubernetes.io/projected/3c1a3f52-3da7-417c-a4e1-a443204576d5-kube-api-access-d6wjz\") pod \"maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn\" (UID: \"3c1a3f52-3da7-417c-a4e1-a443204576d5\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" Oct 30 19:15:00.542749 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.542710 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rzl7n\" (UniqueName: \"kubernetes.io/projected/416b873d-e218-44c4-8b1d-748d2329ae43-kube-api-access-rzl7n\") pod \"maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d\" (UID: \"416b873d-e218-44c4-8b1d-748d2329ae43\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" Oct 30 19:15:00.542911 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.542821 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nb45z\" (UniqueName: \"kubernetes.io/projected/3ff1c8e5-4048-4daa-88df-5287e0e456c6-kube-api-access-nb45z\") pod \"maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s\" (UID: \"3ff1c8e5-4048-4daa-88df-5287e0e456c6\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" Oct 30 19:15:00.547055 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.547033 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" Oct 30 19:15:00.564554 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.564530 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" Oct 30 19:15:00.582045 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.582013 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" Oct 30 19:15:00.840397 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.840349 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" Oct 30 19:15:00.933281 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.933254 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph"] Oct 30 19:15:00.934999 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:15:00.934974 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod290f6845_8233_4640_80a5_d25c79c76339.slice/crio-ef36e7836bdf4cb9b0ae945030426b16a06cb38a331a408c2545918cd297d98f WatchSource:0}: Error finding container ef36e7836bdf4cb9b0ae945030426b16a06cb38a331a408c2545918cd297d98f: Status 404 returned error can't find the container with id ef36e7836bdf4cb9b0ae945030426b16a06cb38a331a408c2545918cd297d98f Oct 30 19:15:00.981764 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:00.981733 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn"] Oct 30 19:15:00.983110 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:15:00.983083 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c1a3f52_3da7_417c_a4e1_a443204576d5.slice/crio-bf6f09bdb37b3e2e4798e208cc69abe9a576e8aabb23ac259069b3f3c2d1cfae WatchSource:0}: Error finding container bf6f09bdb37b3e2e4798e208cc69abe9a576e8aabb23ac259069b3f3c2d1cfae: Status 404 returned error can't find the container with id bf6f09bdb37b3e2e4798e208cc69abe9a576e8aabb23ac259069b3f3c2d1cfae Oct 30 19:15:01.014883 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:01.014856 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d"] Oct 30 19:15:01.015829 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:15:01.015804 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod416b873d_e218_44c4_8b1d_748d2329ae43.slice/crio-02ed9951642a421f2c2166852390ffd3a909f8f249c9573484cbf699b291c814 WatchSource:0}: Error finding container 02ed9951642a421f2c2166852390ffd3a909f8f249c9573484cbf699b291c814: Status 404 returned error can't find the container with id 02ed9951642a421f2c2166852390ffd3a909f8f249c9573484cbf699b291c814 Oct 30 19:15:01.218302 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:01.218279 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s"] Oct 30 19:15:01.219880 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:15:01.219853 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ff1c8e5_4048_4daa_88df_5287e0e456c6.slice/crio-315fbb343bf7407a07bb1139707dde97e506dbda1a5e104ab09f9a9080287d1b WatchSource:0}: Error finding container 315fbb343bf7407a07bb1139707dde97e506dbda1a5e104ab09f9a9080287d1b: Status 404 returned error can't find the container with id 315fbb343bf7407a07bb1139707dde97e506dbda1a5e104ab09f9a9080287d1b Oct 30 19:15:01.526247 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:01.526210 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" event={"ID":"3c1a3f52-3da7-417c-a4e1-a443204576d5","Type":"ContainerStarted","Data":"bf6f09bdb37b3e2e4798e208cc69abe9a576e8aabb23ac259069b3f3c2d1cfae"} Oct 30 19:15:01.527683 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:01.527657 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" event={"ID":"290f6845-8233-4640-80a5-d25c79c76339","Type":"ContainerStarted","Data":"ef36e7836bdf4cb9b0ae945030426b16a06cb38a331a408c2545918cd297d98f"} Oct 30 19:15:01.528935 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:01.528907 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" event={"ID":"3ff1c8e5-4048-4daa-88df-5287e0e456c6","Type":"ContainerStarted","Data":"315fbb343bf7407a07bb1139707dde97e506dbda1a5e104ab09f9a9080287d1b"} Oct 30 19:15:01.530069 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:01.530050 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" event={"ID":"416b873d-e218-44c4-8b1d-748d2329ae43","Type":"ContainerStarted","Data":"02ed9951642a421f2c2166852390ffd3a909f8f249c9573484cbf699b291c814"} Oct 30 19:15:02.392074 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:02.392038 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:05.148450 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:05.148416 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:05.348532 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:05.348495 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:05.367959 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:05.367809 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb"] Oct 30 19:15:05.369178 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:05.369107 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" podUID="7a3b4479-829a-4ac7-8756-52988b12386d" containerName="istio-proxy" containerID="cri-o://5e405111e93065ee9022955a062ce09d0447d3d6b212f9ec067d418c04c55f1d" gracePeriod=30 Oct 30 19:15:05.423305 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:05.422834 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv"] Oct 30 19:15:05.423305 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:05.423080 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" podUID="cf743c6e-7543-408a-b557-cdf069860419" containerName="istio-proxy" containerID="cri-o://5a761ebb66c6a2af78c59dd8a4740811c98092e7495c2579ce52e1e063c830d4" gracePeriod=30 Oct 30 19:15:05.429855 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:05.429433 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw"] Oct 30 19:15:05.429855 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:05.429676 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" podUID="81eabfd4-ebb5-43da-9d2c-03137dada16f" containerName="istio-proxy" containerID="cri-o://33334ea7b59d34dc126dced8a4a04187a8cc4c934a031e477e36a502d33963ed" gracePeriod=30 Oct 30 19:15:05.473732 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:05.473294 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q"] Oct 30 19:15:05.473732 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:05.473548 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" podUID="7b631fda-764d-4310-9ab8-352947f10ee4" containerName="istio-proxy" containerID="cri-o://e463faf08a0b9e0144d9aa9c76416529a1e4c2d7ab0b189421b98a5191b86d07" gracePeriod=30 Oct 30 19:15:05.565197 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:05.565146 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:06.095410 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:06.095277 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:07.127648 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.121043 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:07.313886 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.313857 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:07.495275 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.495244 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:07.596803 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.596764 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" event={"ID":"3ff1c8e5-4048-4daa-88df-5287e0e456c6","Type":"ContainerStarted","Data":"ba2630502ce6e365237c31c3aa2f33034e5a2469a75904e87b92019d5a7fcddc"} Oct 30 19:15:07.598503 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.598476 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" event={"ID":"416b873d-e218-44c4-8b1d-748d2329ae43","Type":"ContainerStarted","Data":"25355f0ce803000a972ab6ca663c899551405f28a4f8f3191139a009ab7cc5f2"} Oct 30 19:15:07.600003 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.599977 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" event={"ID":"3c1a3f52-3da7-417c-a4e1-a443204576d5","Type":"ContainerStarted","Data":"b704890c5a6fd750fe77385c44150260ccd069e8c229a325eb97355ebc00f2da"} Oct 30 19:15:07.618636 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.618608 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4"] Oct 30 19:15:07.622967 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.622948 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4" Oct 30 19:15:07.674886 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.674856 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 19:15:07.680003 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.679973 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4"] Oct 30 19:15:07.729370 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.729304 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" podStartSLOduration=2.029507314 podStartE2EDuration="7.729283567s" totalImagesPullingTime="5.699776253s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:07.725549003 +0000 UTC m=+2178.215676824" watchObservedRunningTime="2026-10-30 19:15:07.729283567 +0000 UTC m=+2178.219411392" Oct 30 19:15:07.730758 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.730708 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcq2t\" (UniqueName: \"kubernetes.io/projected/7d208721-b481-4c47-8ca5-da1f7edab8ac-kube-api-access-fcq2t\") pod \"maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4\" (UID: \"7d208721-b481-4c47-8ca5-da1f7edab8ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4" Oct 30 19:15:07.730906 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.730833 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7d208721-b481-4c47-8ca5-da1f7edab8ac-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4\" (UID: \"7d208721-b481-4c47-8ca5-da1f7edab8ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4" Oct 30 19:15:07.731437 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.731407 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7d208721-b481-4c47-8ca5-da1f7edab8ac-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4\" (UID: \"7d208721-b481-4c47-8ca5-da1f7edab8ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4" Oct 30 19:15:07.832903 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.832870 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7d208721-b481-4c47-8ca5-da1f7edab8ac-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4\" (UID: \"7d208721-b481-4c47-8ca5-da1f7edab8ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4" Oct 30 19:15:07.833113 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.833095 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcq2t\" (UniqueName: \"kubernetes.io/projected/7d208721-b481-4c47-8ca5-da1f7edab8ac-kube-api-access-fcq2t\") pod \"maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4\" (UID: \"7d208721-b481-4c47-8ca5-da1f7edab8ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4" Oct 30 19:15:07.833185 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.833165 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7d208721-b481-4c47-8ca5-da1f7edab8ac-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4\" (UID: \"7d208721-b481-4c47-8ca5-da1f7edab8ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4" Oct 30 19:15:07.833975 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.833944 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7d208721-b481-4c47-8ca5-da1f7edab8ac-maas-api-openshift-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4\" (UID: \"7d208721-b481-4c47-8ca5-da1f7edab8ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4" Oct 30 19:15:07.835997 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.835972 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7d208721-b481-4c47-8ca5-da1f7edab8ac-maas-api-xks-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4\" (UID: \"7d208721-b481-4c47-8ca5-da1f7edab8ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4" Oct 30 19:15:07.846563 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.846515 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" podStartSLOduration=1.947539964 podStartE2EDuration="7.846500086s" totalImagesPullingTime="5.898960122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:07.844787223 +0000 UTC m=+2178.334915041" watchObservedRunningTime="2026-10-30 19:15:07.846500086 +0000 UTC m=+2178.336627902" Oct 30 19:15:07.891313 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.891283 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcq2t\" (UniqueName: \"kubernetes.io/projected/7d208721-b481-4c47-8ca5-da1f7edab8ac-kube-api-access-fcq2t\") pod \"maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4\" (UID: \"7d208721-b481-4c47-8ca5-da1f7edab8ac\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4" Oct 30 19:15:07.963929 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:07.963832 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" podStartSLOduration=2.414786374 podStartE2EDuration="7.963815727s" totalImagesPullingTime="5.549029353s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:07.960945335 +0000 UTC m=+2178.451073155" watchObservedRunningTime="2026-10-30 19:15:07.963815727 +0000 UTC m=+2178.453943591" Oct 30 19:15:08.001852 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:08.001814 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4" Oct 30 19:15:08.378514 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:08.378488 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4"] Oct 30 19:15:08.380008 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:15:08.379984 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d208721_b481_4c47_8ca5_da1f7edab8ac.slice/crio-85117e5369897fb2a47feceacba58197bd3ae154ed39c39fe649af439bb5b6dd WatchSource:0}: Error finding container 85117e5369897fb2a47feceacba58197bd3ae154ed39c39fe649af439bb5b6dd: Status 404 returned error can't find the container with id 85117e5369897fb2a47feceacba58197bd3ae154ed39c39fe649af439bb5b6dd Oct 30 19:15:08.612150 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:08.612110 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" event={"ID":"290f6845-8233-4640-80a5-d25c79c76339","Type":"ContainerStarted","Data":"38f0b6028fc760aeff5f2fe24d0a373ed8fbbdca4b8574ffc47979cbc0bdcef5"} Oct 30 19:15:08.613552 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:08.613523 5894 generic.go:356] "Generic (PLEG): container finished" podID="3ff1c8e5-4048-4daa-88df-5287e0e456c6" containerID="ba2630502ce6e365237c31c3aa2f33034e5a2469a75904e87b92019d5a7fcddc" exitCode=60 Oct 30 19:15:08.613693 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:08.613617 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" event={"ID":"3ff1c8e5-4048-4daa-88df-5287e0e456c6","Type":"ContainerDied","Data":"ba2630502ce6e365237c31c3aa2f33034e5a2469a75904e87b92019d5a7fcddc"} Oct 30 19:15:08.613858 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:08.613843 5894 scope.go:122] "RemoveContainer" containerID="ba2630502ce6e365237c31c3aa2f33034e5a2469a75904e87b92019d5a7fcddc" Oct 30 19:15:08.614994 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:08.614969 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4" event={"ID":"7d208721-b481-4c47-8ca5-da1f7edab8ac","Type":"ContainerStarted","Data":"85117e5369897fb2a47feceacba58197bd3ae154ed39c39fe649af439bb5b6dd"} Oct 30 19:15:08.616646 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:08.616622 5894 generic.go:356] "Generic (PLEG): container finished" podID="416b873d-e218-44c4-8b1d-748d2329ae43" containerID="25355f0ce803000a972ab6ca663c899551405f28a4f8f3191139a009ab7cc5f2" exitCode=60 Oct 30 19:15:08.616742 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:08.616651 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" event={"ID":"416b873d-e218-44c4-8b1d-748d2329ae43","Type":"ContainerDied","Data":"25355f0ce803000a972ab6ca663c899551405f28a4f8f3191139a009ab7cc5f2"} Oct 30 19:15:08.616901 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:08.616885 5894 scope.go:122] "RemoveContainer" containerID="25355f0ce803000a972ab6ca663c899551405f28a4f8f3191139a009ab7cc5f2" Oct 30 19:15:08.618299 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:08.618280 5894 generic.go:356] "Generic (PLEG): container finished" podID="3c1a3f52-3da7-417c-a4e1-a443204576d5" containerID="b704890c5a6fd750fe77385c44150260ccd069e8c229a325eb97355ebc00f2da" exitCode=60 Oct 30 19:15:08.618401 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:08.618376 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" event={"ID":"3c1a3f52-3da7-417c-a4e1-a443204576d5","Type":"ContainerDied","Data":"b704890c5a6fd750fe77385c44150260ccd069e8c229a325eb97355ebc00f2da"} Oct 30 19:15:08.618590 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:08.618540 5894 scope.go:122] "RemoveContainer" containerID="b704890c5a6fd750fe77385c44150260ccd069e8c229a325eb97355ebc00f2da" Oct 30 19:15:08.728179 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:08.728115 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" podStartSLOduration=1.841564735 podStartE2EDuration="8.728097476s" totalImagesPullingTime="6.886532741s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:08.724881499 +0000 UTC m=+2179.215009319" watchObservedRunningTime="2026-10-30 19:15:08.728097476 +0000 UTC m=+2179.218225298" Oct 30 19:15:09.324975 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:09.324945 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:09.626412 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:09.626373 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" event={"ID":"416b873d-e218-44c4-8b1d-748d2329ae43","Type":"ContainerStarted","Data":"d18487a5ce4319aa04df4c10ba4b491c6412bbcf8010af1356942ebc907f799c"} Oct 30 19:15:09.628915 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:09.628885 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" event={"ID":"3c1a3f52-3da7-417c-a4e1-a443204576d5","Type":"ContainerStarted","Data":"52e69ac7b5a1100ccfa68f09f1ec640db627d5d29ec3d493d4d0c80b4faf9c13"} Oct 30 19:15:09.631305 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:09.631277 5894 generic.go:356] "Generic (PLEG): container finished" podID="290f6845-8233-4640-80a5-d25c79c76339" containerID="38f0b6028fc760aeff5f2fe24d0a373ed8fbbdca4b8574ffc47979cbc0bdcef5" exitCode=60 Oct 30 19:15:09.631438 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:09.631367 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" event={"ID":"290f6845-8233-4640-80a5-d25c79c76339","Type":"ContainerDied","Data":"38f0b6028fc760aeff5f2fe24d0a373ed8fbbdca4b8574ffc47979cbc0bdcef5"} Oct 30 19:15:09.631993 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:09.631646 5894 scope.go:122] "RemoveContainer" containerID="38f0b6028fc760aeff5f2fe24d0a373ed8fbbdca4b8574ffc47979cbc0bdcef5" Oct 30 19:15:09.633694 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:09.633673 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" event={"ID":"3ff1c8e5-4048-4daa-88df-5287e0e456c6","Type":"ContainerStarted","Data":"ee6c0c024c1210575159545e7ba428f37d45241cd95a2e4294515419928db202"} Oct 30 19:15:09.635522 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:09.635500 5894 generic.go:356] "Generic (PLEG): container finished" podID="7d208721-b481-4c47-8ca5-da1f7edab8ac" containerID="1ba9a60c1c344d73652c24a20bb96c6818a4469eb7fac33875bf5603305b1f88" exitCode=0 Oct 30 19:15:09.635658 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:09.635548 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4" event={"ID":"7d208721-b481-4c47-8ca5-da1f7edab8ac","Type":"ContainerDied","Data":"1ba9a60c1c344d73652c24a20bb96c6818a4469eb7fac33875bf5603305b1f88"} Oct 30 19:15:10.054762 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.054730 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-4vm2q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:15:10.054911 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.054787 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" podUID="7b631fda-764d-4310-9ab8-352947f10ee4" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:15:10.652996 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.652957 5894 generic.go:356] "Generic (PLEG): container finished" podID="7a3b4479-829a-4ac7-8756-52988b12386d" containerID="5e405111e93065ee9022955a062ce09d0447d3d6b212f9ec067d418c04c55f1d" exitCode=0 Oct 30 19:15:10.653325 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.653089 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" event={"ID":"7a3b4479-829a-4ac7-8756-52988b12386d","Type":"ContainerDied","Data":"5e405111e93065ee9022955a062ce09d0447d3d6b212f9ec067d418c04c55f1d"} Oct 30 19:15:10.667708 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.655143 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" event={"ID":"290f6845-8233-4640-80a5-d25c79c76339","Type":"ContainerStarted","Data":"31e81901c69da8f7a3950173418aca80ebc7babd0b80b01d6321a97bca771123"} Oct 30 19:15:10.667708 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.658461 5894 generic.go:356] "Generic (PLEG): container finished" podID="3ff1c8e5-4048-4daa-88df-5287e0e456c6" containerID="ee6c0c024c1210575159545e7ba428f37d45241cd95a2e4294515419928db202" exitCode=60 Oct 30 19:15:10.667708 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.658620 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" event={"ID":"3ff1c8e5-4048-4daa-88df-5287e0e456c6","Type":"ContainerDied","Data":"ee6c0c024c1210575159545e7ba428f37d45241cd95a2e4294515419928db202"} Oct 30 19:15:10.667708 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.658668 5894 scope.go:122] "RemoveContainer" containerID="ba2630502ce6e365237c31c3aa2f33034e5a2469a75904e87b92019d5a7fcddc" Oct 30 19:15:10.667708 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.659161 5894 scope.go:122] "RemoveContainer" containerID="ee6c0c024c1210575159545e7ba428f37d45241cd95a2e4294515419928db202" Oct 30 19:15:10.667708 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:15:10.659541 5894 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s_odh-ai-gateway-infra(3ff1c8e5-4048-4daa-88df-5287e0e456c6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" podUID="3ff1c8e5-4048-4daa-88df-5287e0e456c6" Oct 30 19:15:10.667708 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.664287 5894 generic.go:356] "Generic (PLEG): container finished" podID="416b873d-e218-44c4-8b1d-748d2329ae43" containerID="d18487a5ce4319aa04df4c10ba4b491c6412bbcf8010af1356942ebc907f799c" exitCode=60 Oct 30 19:15:10.667708 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.664388 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" event={"ID":"416b873d-e218-44c4-8b1d-748d2329ae43","Type":"ContainerDied","Data":"d18487a5ce4319aa04df4c10ba4b491c6412bbcf8010af1356942ebc907f799c"} Oct 30 19:15:10.667708 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.664812 5894 scope.go:122] "RemoveContainer" containerID="d18487a5ce4319aa04df4c10ba4b491c6412bbcf8010af1356942ebc907f799c" Oct 30 19:15:10.667708 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:15:10.665133 5894 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d_odh-ai-gateway-infra(416b873d-e218-44c4-8b1d-748d2329ae43)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" podUID="416b873d-e218-44c4-8b1d-748d2329ae43" Oct 30 19:15:10.668658 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.668631 5894 generic.go:356] "Generic (PLEG): container finished" podID="3c1a3f52-3da7-417c-a4e1-a443204576d5" containerID="52e69ac7b5a1100ccfa68f09f1ec640db627d5d29ec3d493d4d0c80b4faf9c13" exitCode=60 Oct 30 19:15:10.668754 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.668736 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" event={"ID":"3c1a3f52-3da7-417c-a4e1-a443204576d5","Type":"ContainerDied","Data":"52e69ac7b5a1100ccfa68f09f1ec640db627d5d29ec3d493d4d0c80b4faf9c13"} Oct 30 19:15:10.669149 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.669131 5894 scope.go:122] "RemoveContainer" containerID="52e69ac7b5a1100ccfa68f09f1ec640db627d5d29ec3d493d4d0c80b4faf9c13" Oct 30 19:15:10.669484 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:15:10.669462 5894 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn_odh-ai-gateway-infra(3c1a3f52-3da7-417c-a4e1-a443204576d5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" podUID="3c1a3f52-3da7-417c-a4e1-a443204576d5" Oct 30 19:15:10.682591 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.672794 5894 generic.go:356] "Generic (PLEG): container finished" podID="81eabfd4-ebb5-43da-9d2c-03137dada16f" containerID="33334ea7b59d34dc126dced8a4a04187a8cc4c934a031e477e36a502d33963ed" exitCode=0 Oct 30 19:15:10.682591 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.672883 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" event={"ID":"81eabfd4-ebb5-43da-9d2c-03137dada16f","Type":"ContainerDied","Data":"33334ea7b59d34dc126dced8a4a04187a8cc4c934a031e477e36a502d33963ed"} Oct 30 19:15:10.682591 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.676675 5894 generic.go:356] "Generic (PLEG): container finished" podID="7b631fda-764d-4310-9ab8-352947f10ee4" containerID="e463faf08a0b9e0144d9aa9c76416529a1e4c2d7ab0b189421b98a5191b86d07" exitCode=0 Oct 30 19:15:10.682591 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.676756 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" event={"ID":"7b631fda-764d-4310-9ab8-352947f10ee4","Type":"ContainerDied","Data":"e463faf08a0b9e0144d9aa9c76416529a1e4c2d7ab0b189421b98a5191b86d07"} Oct 30 19:15:10.682591 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.678537 5894 generic.go:356] "Generic (PLEG): container finished" podID="cf743c6e-7543-408a-b557-cdf069860419" containerID="5a761ebb66c6a2af78c59dd8a4740811c98092e7495c2579ce52e1e063c830d4" exitCode=0 Oct 30 19:15:10.688974 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.688941 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" event={"ID":"cf743c6e-7543-408a-b557-cdf069860419","Type":"ContainerDied","Data":"5a761ebb66c6a2af78c59dd8a4740811c98092e7495c2579ce52e1e063c830d4"} Oct 30 19:15:10.705333 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.701536 5894 scope.go:122] "RemoveContainer" containerID="25355f0ce803000a972ab6ca663c899551405f28a4f8f3191139a009ab7cc5f2" Oct 30 19:15:10.812468 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.812440 5894 scope.go:122] "RemoveContainer" containerID="b704890c5a6fd750fe77385c44150260ccd069e8c229a325eb97355ebc00f2da" Oct 30 19:15:10.812999 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.812970 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:15:10.969973 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.969812 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:15:10.973149 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.973124 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:15:10.977179 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.977159 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:15:10.981736 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.981708 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-istio-envoy\") pod \"7a3b4479-829a-4ac7-8756-52988b12386d\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " Oct 30 19:15:10.981847 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.981765 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-workload-socket\") pod \"7a3b4479-829a-4ac7-8756-52988b12386d\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " Oct 30 19:15:10.981847 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.981821 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7a3b4479-829a-4ac7-8756-52988b12386d-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a3b4479-829a-4ac7-8756-52988b12386d-istiod-ca-cert\") pod \"7a3b4479-829a-4ac7-8756-52988b12386d\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " Oct 30 19:15:10.981954 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.981874 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-istio-data\") pod \"7a3b4479-829a-4ac7-8756-52988b12386d\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " Oct 30 19:15:10.981954 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.981917 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-credential-socket\") pod \"7a3b4479-829a-4ac7-8756-52988b12386d\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " Oct 30 19:15:10.982077 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.981970 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a3b4479-829a-4ac7-8756-52988b12386d-istio-token\" (UniqueName: \"kubernetes.io/projected/7a3b4479-829a-4ac7-8756-52988b12386d-istio-token\") pod \"7a3b4479-829a-4ac7-8756-52988b12386d\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " Oct 30 19:15:10.982077 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.982006 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7a3b4479-829a-4ac7-8756-52988b12386d-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a3b4479-829a-4ac7-8756-52988b12386d-istio-podinfo\") pod \"7a3b4479-829a-4ac7-8756-52988b12386d\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " Oct 30 19:15:10.982077 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.982032 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-workload-socket" pod "7a3b4479-829a-4ac7-8756-52988b12386d" (UID: "7a3b4479-829a-4ac7-8756-52988b12386d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:10.982229 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.982075 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7a3b4479-829a-4ac7-8756-52988b12386d-kube-api-access-tvhfn\" (UniqueName: \"kubernetes.io/projected/7a3b4479-829a-4ac7-8756-52988b12386d-kube-api-access-tvhfn\") pod \"7a3b4479-829a-4ac7-8756-52988b12386d\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " Oct 30 19:15:10.982229 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.982154 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-workload-certs\") pod \"7a3b4479-829a-4ac7-8756-52988b12386d\" (UID: \"7a3b4479-829a-4ac7-8756-52988b12386d\") " Oct 30 19:15:10.982336 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.982278 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-istio-data" pod "7a3b4479-829a-4ac7-8756-52988b12386d" (UID: "7a3b4479-829a-4ac7-8756-52988b12386d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:10.982628 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.982603 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7a3b4479-829a-4ac7-8756-52988b12386d-istiod-ca-cert" pod "7a3b4479-829a-4ac7-8756-52988b12386d" (UID: "7a3b4479-829a-4ac7-8756-52988b12386d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:15:10.982851 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.982828 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-workload-certs" pod "7a3b4479-829a-4ac7-8756-52988b12386d" (UID: "7a3b4479-829a-4ac7-8756-52988b12386d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:10.982981 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.982960 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:10.983056 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.982989 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7a3b4479-829a-4ac7-8756-52988b12386d-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:10.983056 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.983003 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:10.983056 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.983017 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:10.983246 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.983226 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-credential-socket" pod "7a3b4479-829a-4ac7-8756-52988b12386d" (UID: "7a3b4479-829a-4ac7-8756-52988b12386d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:10.985358 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.985326 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a3b4479-829a-4ac7-8756-52988b12386d-istio-token" pod "7a3b4479-829a-4ac7-8756-52988b12386d" (UID: "7a3b4479-829a-4ac7-8756-52988b12386d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:15:10.986256 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.986228 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7a3b4479-829a-4ac7-8756-52988b12386d-kube-api-access-tvhfn" pod "7a3b4479-829a-4ac7-8756-52988b12386d" (UID: "7a3b4479-829a-4ac7-8756-52988b12386d"). InnerVolumeSpecName "kube-api-access-tvhfn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:15:10.987500 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.987478 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-istio-envoy" pod "7a3b4479-829a-4ac7-8756-52988b12386d" (UID: "7a3b4479-829a-4ac7-8756-52988b12386d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:10.989797 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:10.989768 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7a3b4479-829a-4ac7-8756-52988b12386d-istio-podinfo" pod "7a3b4479-829a-4ac7-8756-52988b12386d" (UID: "7a3b4479-829a-4ac7-8756-52988b12386d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:15:11.029312 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.029283 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4" Oct 30 19:15:11.083843 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.083808 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-credential-socket\") pod \"7b631fda-764d-4310-9ab8-352947f10ee4\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " Oct 30 19:15:11.084051 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.083875 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/cf743c6e-7543-408a-b557-cdf069860419-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf743c6e-7543-408a-b557-cdf069860419-istio-podinfo\") pod \"cf743c6e-7543-408a-b557-cdf069860419\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " Oct 30 19:15:11.084051 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.083908 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-token\" (UniqueName: \"kubernetes.io/projected/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-token\") pod \"81eabfd4-ebb5-43da-9d2c-03137dada16f\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " Oct 30 19:15:11.084051 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.083929 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-envoy\") pod \"81eabfd4-ebb5-43da-9d2c-03137dada16f\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " Oct 30 19:15:11.084051 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.083953 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7b631fda-764d-4310-9ab8-352947f10ee4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b631fda-764d-4310-9ab8-352947f10ee4-istiod-ca-cert\") pod \"7b631fda-764d-4310-9ab8-352947f10ee4\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " Oct 30 19:15:11.084051 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.083985 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-istio-data\") pod \"cf743c6e-7543-408a-b557-cdf069860419\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " Oct 30 19:15:11.084051 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084043 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/81eabfd4-ebb5-43da-9d2c-03137dada16f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81eabfd4-ebb5-43da-9d2c-03137dada16f-istiod-ca-cert\") pod \"81eabfd4-ebb5-43da-9d2c-03137dada16f\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " Oct 30 19:15:11.084362 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084126 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/cf743c6e-7543-408a-b557-cdf069860419-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf743c6e-7543-408a-b557-cdf069860419-istiod-ca-cert\") pod \"cf743c6e-7543-408a-b557-cdf069860419\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " Oct 30 19:15:11.084362 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084165 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-workload-socket\") pod \"7b631fda-764d-4310-9ab8-352947f10ee4\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " Oct 30 19:15:11.084362 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084193 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7b631fda-764d-4310-9ab8-352947f10ee4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b631fda-764d-4310-9ab8-352947f10ee4-istio-podinfo\") pod \"7b631fda-764d-4310-9ab8-352947f10ee4\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " Oct 30 19:15:11.084362 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084227 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-podinfo\") pod \"81eabfd4-ebb5-43da-9d2c-03137dada16f\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " Oct 30 19:15:11.084362 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084260 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-istio-data\") pod \"7b631fda-764d-4310-9ab8-352947f10ee4\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " Oct 30 19:15:11.084362 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084159 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-credential-socket" pod "7b631fda-764d-4310-9ab8-352947f10ee4" (UID: "7b631fda-764d-4310-9ab8-352947f10ee4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:11.084362 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084306 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7b631fda-764d-4310-9ab8-352947f10ee4-kube-api-access-lwvxw\" (UniqueName: \"kubernetes.io/projected/7b631fda-764d-4310-9ab8-352947f10ee4-kube-api-access-lwvxw\") pod \"7b631fda-764d-4310-9ab8-352947f10ee4\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " Oct 30 19:15:11.084362 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084337 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-credential-socket\") pod \"81eabfd4-ebb5-43da-9d2c-03137dada16f\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " Oct 30 19:15:11.084788 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084389 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/81eabfd4-ebb5-43da-9d2c-03137dada16f-kube-api-access-6p8hh\" (UniqueName: \"kubernetes.io/projected/81eabfd4-ebb5-43da-9d2c-03137dada16f-kube-api-access-6p8hh\") pod \"81eabfd4-ebb5-43da-9d2c-03137dada16f\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " Oct 30 19:15:11.084788 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084413 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7b631fda-764d-4310-9ab8-352947f10ee4-istiod-ca-cert" pod "7b631fda-764d-4310-9ab8-352947f10ee4" (UID: "7b631fda-764d-4310-9ab8-352947f10ee4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:15:11.084788 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084447 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-workload-socket\") pod \"cf743c6e-7543-408a-b557-cdf069860419\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " Oct 30 19:15:11.084788 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084503 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-istio-envoy\") pod \"cf743c6e-7543-408a-b557-cdf069860419\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " Oct 30 19:15:11.084788 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084533 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-workload-certs\") pod \"cf743c6e-7543-408a-b557-cdf069860419\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " Oct 30 19:15:11.084788 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084564 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf743c6e-7543-408a-b557-cdf069860419-kube-api-access-vjnw8\" (UniqueName: \"kubernetes.io/projected/cf743c6e-7543-408a-b557-cdf069860419-kube-api-access-vjnw8\") pod \"cf743c6e-7543-408a-b557-cdf069860419\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " Oct 30 19:15:11.084788 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084615 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81eabfd4-ebb5-43da-9d2c-03137dada16f-istiod-ca-cert" pod "81eabfd4-ebb5-43da-9d2c-03137dada16f" (UID: "81eabfd4-ebb5-43da-9d2c-03137dada16f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:15:11.084788 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084695 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-credential-socket\") pod \"cf743c6e-7543-408a-b557-cdf069860419\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " Oct 30 19:15:11.084788 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084736 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-workload-certs\") pod \"81eabfd4-ebb5-43da-9d2c-03137dada16f\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " Oct 30 19:15:11.084788 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084775 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-istio-envoy\") pod \"7b631fda-764d-4310-9ab8-352947f10ee4\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " Oct 30 19:15:11.085329 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084809 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-workload-socket\") pod \"81eabfd4-ebb5-43da-9d2c-03137dada16f\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " Oct 30 19:15:11.085329 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084842 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/cf743c6e-7543-408a-b557-cdf069860419-istio-token\" (UniqueName: \"kubernetes.io/projected/cf743c6e-7543-408a-b557-cdf069860419-istio-token\") pod \"cf743c6e-7543-408a-b557-cdf069860419\" (UID: \"cf743c6e-7543-408a-b557-cdf069860419\") " Oct 30 19:15:11.085329 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084735 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-workload-socket" pod "7b631fda-764d-4310-9ab8-352947f10ee4" (UID: "7b631fda-764d-4310-9ab8-352947f10ee4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:11.085329 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084665 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cf743c6e-7543-408a-b557-cdf069860419-istiod-ca-cert" pod "cf743c6e-7543-408a-b557-cdf069860419" (UID: "cf743c6e-7543-408a-b557-cdf069860419"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:15:11.085329 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084838 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-istio-data" pod "cf743c6e-7543-408a-b557-cdf069860419" (UID: "cf743c6e-7543-408a-b557-cdf069860419"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:11.085329 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084886 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7b631fda-764d-4310-9ab8-352947f10ee4-istio-token\" (UniqueName: \"kubernetes.io/projected/7b631fda-764d-4310-9ab8-352947f10ee4-istio-token\") pod \"7b631fda-764d-4310-9ab8-352947f10ee4\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " Oct 30 19:15:11.085329 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084914 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-data\") pod \"81eabfd4-ebb5-43da-9d2c-03137dada16f\" (UID: \"81eabfd4-ebb5-43da-9d2c-03137dada16f\") " Oct 30 19:15:11.085329 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084968 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-workload-socket" pod "cf743c6e-7543-408a-b557-cdf069860419" (UID: "cf743c6e-7543-408a-b557-cdf069860419"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:11.085329 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.084981 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-workload-certs\") pod \"7b631fda-764d-4310-9ab8-352947f10ee4\" (UID: \"7b631fda-764d-4310-9ab8-352947f10ee4\") " Oct 30 19:15:11.085329 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.085065 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-credential-socket" pod "cf743c6e-7543-408a-b557-cdf069860419" (UID: "cf743c6e-7543-408a-b557-cdf069860419"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:11.085329 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.085271 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-workload-certs" pod "81eabfd4-ebb5-43da-9d2c-03137dada16f" (UID: "81eabfd4-ebb5-43da-9d2c-03137dada16f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:11.085917 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.085690 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-workload-certs" pod "cf743c6e-7543-408a-b557-cdf069860419" (UID: "cf743c6e-7543-408a-b557-cdf069860419"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:11.086301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.086020 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cf743c6e-7543-408a-b557-cdf069860419-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.086301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.086048 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.086301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.086064 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.086301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.086079 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.086301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.086094 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.086301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.086110 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7a3b4479-829a-4ac7-8756-52988b12386d-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.086301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.086125 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.086301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.086138 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.086301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.086152 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7a3b4479-829a-4ac7-8756-52988b12386d-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.086301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.086165 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7a3b4479-829a-4ac7-8756-52988b12386d-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.086301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.086180 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvhfn\" (UniqueName: \"kubernetes.io/projected/7a3b4479-829a-4ac7-8756-52988b12386d-kube-api-access-tvhfn\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.086301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.086196 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.086301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.086210 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7b631fda-764d-4310-9ab8-352947f10ee4-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.086301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.086227 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/81eabfd4-ebb5-43da-9d2c-03137dada16f-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.087234 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.086374 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-istio-data" pod "7b631fda-764d-4310-9ab8-352947f10ee4" (UID: "7b631fda-764d-4310-9ab8-352947f10ee4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:11.087234 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.086607 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-workload-socket" pod "81eabfd4-ebb5-43da-9d2c-03137dada16f" (UID: "81eabfd4-ebb5-43da-9d2c-03137dada16f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:11.087234 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.086630 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-data" pod "81eabfd4-ebb5-43da-9d2c-03137dada16f" (UID: "81eabfd4-ebb5-43da-9d2c-03137dada16f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:11.087234 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.086828 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-credential-socket" pod "81eabfd4-ebb5-43da-9d2c-03137dada16f" (UID: "81eabfd4-ebb5-43da-9d2c-03137dada16f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:11.087234 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.086957 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-workload-certs" pod "7b631fda-764d-4310-9ab8-352947f10ee4" (UID: "7b631fda-764d-4310-9ab8-352947f10ee4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:11.087740 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.087715 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-envoy" pod "81eabfd4-ebb5-43da-9d2c-03137dada16f" (UID: "81eabfd4-ebb5-43da-9d2c-03137dada16f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:11.088440 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.088401 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7b631fda-764d-4310-9ab8-352947f10ee4-istio-podinfo" pod "7b631fda-764d-4310-9ab8-352947f10ee4" (UID: "7b631fda-764d-4310-9ab8-352947f10ee4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:15:11.088873 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.088536 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-istio-envoy" pod "7b631fda-764d-4310-9ab8-352947f10ee4" (UID: "7b631fda-764d-4310-9ab8-352947f10ee4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:11.088873 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.088700 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-istio-envoy" pod "cf743c6e-7543-408a-b557-cdf069860419" (UID: "cf743c6e-7543-408a-b557-cdf069860419"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:15:11.088873 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.088777 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cf743c6e-7543-408a-b557-cdf069860419-istio-podinfo" pod "cf743c6e-7543-408a-b557-cdf069860419" (UID: "cf743c6e-7543-408a-b557-cdf069860419"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:15:11.088873 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.088841 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf743c6e-7543-408a-b557-cdf069860419-kube-api-access-vjnw8" pod "cf743c6e-7543-408a-b557-cdf069860419" (UID: "cf743c6e-7543-408a-b557-cdf069860419"). InnerVolumeSpecName "kube-api-access-vjnw8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:15:11.089534 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.089496 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b631fda-764d-4310-9ab8-352947f10ee4-kube-api-access-lwvxw" pod "7b631fda-764d-4310-9ab8-352947f10ee4" (UID: "7b631fda-764d-4310-9ab8-352947f10ee4"). InnerVolumeSpecName "kube-api-access-lwvxw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:15:11.089674 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.089537 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-token" pod "81eabfd4-ebb5-43da-9d2c-03137dada16f" (UID: "81eabfd4-ebb5-43da-9d2c-03137dada16f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:15:11.089889 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.089865 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-podinfo" pod "81eabfd4-ebb5-43da-9d2c-03137dada16f" (UID: "81eabfd4-ebb5-43da-9d2c-03137dada16f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:15:11.090167 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.090146 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81eabfd4-ebb5-43da-9d2c-03137dada16f-kube-api-access-6p8hh" pod "81eabfd4-ebb5-43da-9d2c-03137dada16f" (UID: "81eabfd4-ebb5-43da-9d2c-03137dada16f"). InnerVolumeSpecName "kube-api-access-6p8hh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:15:11.090246 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.090179 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cf743c6e-7543-408a-b557-cdf069860419-istio-token" pod "cf743c6e-7543-408a-b557-cdf069860419" (UID: "cf743c6e-7543-408a-b557-cdf069860419"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:15:11.090283 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.090254 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b631fda-764d-4310-9ab8-352947f10ee4-istio-token" pod "7b631fda-764d-4310-9ab8-352947f10ee4" (UID: "7b631fda-764d-4310-9ab8-352947f10ee4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:15:11.187619 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.187558 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d208721-b481-4c47-8ca5-da1f7edab8ac-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7d208721-b481-4c47-8ca5-da1f7edab8ac-maas-api-openshift-ca\") pod \"7d208721-b481-4c47-8ca5-da1f7edab8ac\" (UID: \"7d208721-b481-4c47-8ca5-da1f7edab8ac\") " Oct 30 19:15:11.187808 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.187662 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d208721-b481-4c47-8ca5-da1f7edab8ac-kube-api-access-fcq2t\" (UniqueName: \"kubernetes.io/projected/7d208721-b481-4c47-8ca5-da1f7edab8ac-kube-api-access-fcq2t\") pod \"7d208721-b481-4c47-8ca5-da1f7edab8ac\" (UID: \"7d208721-b481-4c47-8ca5-da1f7edab8ac\") " Oct 30 19:15:11.187867 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.187835 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7d208721-b481-4c47-8ca5-da1f7edab8ac-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7d208721-b481-4c47-8ca5-da1f7edab8ac-maas-api-xks-ca\") pod \"7d208721-b481-4c47-8ca5-da1f7edab8ac\" (UID: \"7d208721-b481-4c47-8ca5-da1f7edab8ac\") " Oct 30 19:15:11.187921 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.187889 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d208721-b481-4c47-8ca5-da1f7edab8ac-maas-api-openshift-ca" pod "7d208721-b481-4c47-8ca5-da1f7edab8ac" (UID: "7d208721-b481-4c47-8ca5-da1f7edab8ac"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:15:11.188294 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.188271 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7b631fda-764d-4310-9ab8-352947f10ee4-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.188372 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.188298 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.188372 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.188309 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.188372 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.188318 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lwvxw\" (UniqueName: \"kubernetes.io/projected/7b631fda-764d-4310-9ab8-352947f10ee4-kube-api-access-lwvxw\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.188372 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.188329 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.188372 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.188342 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6p8hh\" (UniqueName: \"kubernetes.io/projected/81eabfd4-ebb5-43da-9d2c-03137dada16f-kube-api-access-6p8hh\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.188372 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.188356 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.188372 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.188366 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjnw8\" (UniqueName: \"kubernetes.io/projected/cf743c6e-7543-408a-b557-cdf069860419-kube-api-access-vjnw8\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.188372 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.188375 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.188660 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.188383 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.188660 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.188392 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cf743c6e-7543-408a-b557-cdf069860419-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.188660 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.188402 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/7d208721-b481-4c47-8ca5-da1f7edab8ac-maas-api-openshift-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.188660 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.188411 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7b631fda-764d-4310-9ab8-352947f10ee4-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.188660 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.188420 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.188660 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.188432 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7b631fda-764d-4310-9ab8-352947f10ee4-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.188660 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.188440 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cf743c6e-7543-408a-b557-cdf069860419-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.188660 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.188449 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.188660 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.188457 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/81eabfd4-ebb5-43da-9d2c-03137dada16f-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.188660 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.188466 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cf743c6e-7543-408a-b557-cdf069860419-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.189777 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.189754 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d208721-b481-4c47-8ca5-da1f7edab8ac-maas-api-xks-ca" pod "7d208721-b481-4c47-8ca5-da1f7edab8ac" (UID: "7d208721-b481-4c47-8ca5-da1f7edab8ac"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:15:11.189838 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.189821 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d208721-b481-4c47-8ca5-da1f7edab8ac-kube-api-access-fcq2t" pod "7d208721-b481-4c47-8ca5-da1f7edab8ac" (UID: "7d208721-b481-4c47-8ca5-da1f7edab8ac"). InnerVolumeSpecName "kube-api-access-fcq2t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:15:11.289764 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.289731 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcq2t\" (UniqueName: \"kubernetes.io/projected/7d208721-b481-4c47-8ca5-da1f7edab8ac-kube-api-access-fcq2t\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.289764 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.289763 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/7d208721-b481-4c47-8ca5-da1f7edab8ac-maas-api-xks-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:11.687490 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.687460 5894 generic.go:356] "Generic (PLEG): container finished" podID="290f6845-8233-4640-80a5-d25c79c76339" containerID="31e81901c69da8f7a3950173418aca80ebc7babd0b80b01d6321a97bca771123" exitCode=60 Oct 30 19:15:11.687862 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.687544 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" event={"ID":"290f6845-8233-4640-80a5-d25c79c76339","Type":"ContainerDied","Data":"31e81901c69da8f7a3950173418aca80ebc7babd0b80b01d6321a97bca771123"} Oct 30 19:15:11.687862 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.687630 5894 scope.go:122] "RemoveContainer" containerID="38f0b6028fc760aeff5f2fe24d0a373ed8fbbdca4b8574ffc47979cbc0bdcef5" Oct 30 19:15:11.687862 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.687840 5894 scope.go:122] "RemoveContainer" containerID="31e81901c69da8f7a3950173418aca80ebc7babd0b80b01d6321a97bca771123" Oct 30 19:15:11.688228 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:15:11.688200 5894 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph_odh-ai-gateway-infra(290f6845-8233-4640-80a5-d25c79c76339)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" podUID="290f6845-8233-4640-80a5-d25c79c76339" Oct 30 19:15:11.689669 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.689649 5894 scope.go:122] "RemoveContainer" containerID="ee6c0c024c1210575159545e7ba428f37d45241cd95a2e4294515419928db202" Oct 30 19:15:11.689926 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:15:11.689908 5894 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s_odh-ai-gateway-infra(3ff1c8e5-4048-4daa-88df-5287e0e456c6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" podUID="3ff1c8e5-4048-4daa-88df-5287e0e456c6" Oct 30 19:15:11.691122 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.691101 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4" event={"ID":"7d208721-b481-4c47-8ca5-da1f7edab8ac","Type":"ContainerDied","Data":"85117e5369897fb2a47feceacba58197bd3ae154ed39c39fe649af439bb5b6dd"} Oct 30 19:15:11.691223 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.691140 5894 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="85117e5369897fb2a47feceacba58197bd3ae154ed39c39fe649af439bb5b6dd" Oct 30 19:15:11.691223 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.691105 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4" Oct 30 19:15:11.693150 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.693134 5894 scope.go:122] "RemoveContainer" containerID="d18487a5ce4319aa04df4c10ba4b491c6412bbcf8010af1356942ebc907f799c" Oct 30 19:15:11.693504 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:15:11.693482 5894 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d_odh-ai-gateway-infra(416b873d-e218-44c4-8b1d-748d2329ae43)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" podUID="416b873d-e218-44c4-8b1d-748d2329ae43" Oct 30 19:15:11.695340 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.695322 5894 scope.go:122] "RemoveContainer" containerID="52e69ac7b5a1100ccfa68f09f1ec640db627d5d29ec3d493d4d0c80b4faf9c13" Oct 30 19:15:11.695499 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:15:11.695484 5894 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn_odh-ai-gateway-infra(3c1a3f52-3da7-417c-a4e1-a443204576d5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" podUID="3c1a3f52-3da7-417c-a4e1-a443204576d5" Oct 30 19:15:11.696807 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.696789 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" event={"ID":"81eabfd4-ebb5-43da-9d2c-03137dada16f","Type":"ContainerDied","Data":"d1bafd976f340ed65d908c800fbd9711200f4f351af027edfdceeb5e1fb109ef"} Oct 30 19:15:11.696863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.696852 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw" Oct 30 19:15:11.698522 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.698491 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" Oct 30 19:15:11.698640 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.698520 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q" event={"ID":"7b631fda-764d-4310-9ab8-352947f10ee4","Type":"ContainerDied","Data":"4e17b02e70e5eab54d14523fa06a9b8fdc52e71818372415f566a600c1fa136c"} Oct 30 19:15:11.705679 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.705660 5894 scope.go:122] "RemoveContainer" containerID="33334ea7b59d34dc126dced8a4a04187a8cc4c934a031e477e36a502d33963ed" Oct 30 19:15:11.708303 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.708273 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" Oct 30 19:15:11.708303 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.708284 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv" event={"ID":"cf743c6e-7543-408a-b557-cdf069860419","Type":"ContainerDied","Data":"99bf07fc6ecf5c267134593004506dcf520d5bdf0c60d4e506ba6c6e48db7721"} Oct 30 19:15:11.709865 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.709844 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" event={"ID":"7a3b4479-829a-4ac7-8756-52988b12386d","Type":"ContainerDied","Data":"eb3a0b9d118bdfd3ffc41eb5a02aba874feb0935239db4cc3f88af2e6b6c30b1"} Oct 30 19:15:11.709956 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.709901 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb" Oct 30 19:15:11.722343 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.722315 5894 scope.go:122] "RemoveContainer" containerID="e463faf08a0b9e0144d9aa9c76416529a1e4c2d7ab0b189421b98a5191b86d07" Oct 30 19:15:11.735083 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.735061 5894 scope.go:122] "RemoveContainer" containerID="5a761ebb66c6a2af78c59dd8a4740811c98092e7495c2579ce52e1e063c830d4" Oct 30 19:15:11.746975 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:11.746949 5894 scope.go:122] "RemoveContainer" containerID="5e405111e93065ee9022955a062ce09d0447d3d6b212f9ec067d418c04c55f1d" Oct 30 19:15:12.427326 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:12.427286 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:12.428340 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:12.428316 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb"] Oct 30 19:15:12.431810 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:12.431790 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-rsntb"] Oct 30 19:15:12.540056 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:12.540021 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw"] Oct 30 19:15:12.565555 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:12.565515 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-jk6fw"] Oct 30 19:15:12.629530 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:12.629492 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:12.699957 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:12.698445 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q"] Oct 30 19:15:12.703329 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:12.703304 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-4vm2q"] Oct 30 19:15:12.727216 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:12.727183 5894 scope.go:122] "RemoveContainer" containerID="31e81901c69da8f7a3950173418aca80ebc7babd0b80b01d6321a97bca771123" Oct 30 19:15:12.727492 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:15:12.727468 5894 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph_odh-ai-gateway-infra(290f6845-8233-4640-80a5-d25c79c76339)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" podUID="290f6845-8233-4640-80a5-d25c79c76339" Oct 30 19:15:12.792274 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:12.792235 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv"] Oct 30 19:15:12.797777 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:12.797751 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-2vwtv"] Oct 30 19:15:13.651371 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:13.651337 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7a3b4479-829a-4ac7-8756-52988b12386d" path="/var/lib/kubelet/pods/7a3b4479-829a-4ac7-8756-52988b12386d/volumes" Oct 30 19:15:13.651837 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:13.651822 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7b631fda-764d-4310-9ab8-352947f10ee4" path="/var/lib/kubelet/pods/7b631fda-764d-4310-9ab8-352947f10ee4/volumes" Oct 30 19:15:13.652265 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:13.652247 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="81eabfd4-ebb5-43da-9d2c-03137dada16f" path="/var/lib/kubelet/pods/81eabfd4-ebb5-43da-9d2c-03137dada16f/volumes" Oct 30 19:15:13.653107 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:13.653084 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="cf743c6e-7543-408a-b557-cdf069860419" path="/var/lib/kubelet/pods/cf743c6e-7543-408a-b557-cdf069860419/volumes" Oct 30 19:15:14.489189 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:14.489156 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:14.681254 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:14.681220 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:17.814727 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:17.814684 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:19.110369 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.110323 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d"] Oct 30 19:15:19.111236 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.111214 5894 cpu_manager.go:414] "containerMap: removing container" podUID="81eabfd4-ebb5-43da-9d2c-03137dada16f" containerName="istio-proxy" Oct 30 19:15:19.111236 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.111236 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="81eabfd4-ebb5-43da-9d2c-03137dada16f" containerName="istio-proxy" Oct 30 19:15:19.111417 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.111255 5894 cpu_manager.go:414] "containerMap: removing container" podUID="7d208721-b481-4c47-8ca5-da1f7edab8ac" containerName="revoke-keys" Oct 30 19:15:19.111417 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.111262 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d208721-b481-4c47-8ca5-da1f7edab8ac" containerName="revoke-keys" Oct 30 19:15:19.111417 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.111273 5894 cpu_manager.go:414] "containerMap: removing container" podUID="7b631fda-764d-4310-9ab8-352947f10ee4" containerName="istio-proxy" Oct 30 19:15:19.111417 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.111280 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7b631fda-764d-4310-9ab8-352947f10ee4" containerName="istio-proxy" Oct 30 19:15:19.111417 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.111345 5894 cpu_manager.go:414] "containerMap: removing container" podUID="cf743c6e-7543-408a-b557-cdf069860419" containerName="istio-proxy" Oct 30 19:15:19.111417 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.111353 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="cf743c6e-7543-408a-b557-cdf069860419" containerName="istio-proxy" Oct 30 19:15:19.111417 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.111364 5894 cpu_manager.go:414] "containerMap: removing container" podUID="7a3b4479-829a-4ac7-8756-52988b12386d" containerName="istio-proxy" Oct 30 19:15:19.111417 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.111371 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7a3b4479-829a-4ac7-8756-52988b12386d" containerName="istio-proxy" Oct 30 19:15:19.111832 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.111537 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="81eabfd4-ebb5-43da-9d2c-03137dada16f" containerName="istio-proxy" Oct 30 19:15:19.111832 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.111549 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d208721-b481-4c47-8ca5-da1f7edab8ac" containerName="revoke-keys" Oct 30 19:15:19.111832 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.111559 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="7b631fda-764d-4310-9ab8-352947f10ee4" containerName="istio-proxy" Oct 30 19:15:19.111832 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.111596 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="cf743c6e-7543-408a-b557-cdf069860419" containerName="istio-proxy" Oct 30 19:15:19.111832 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.111605 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="7a3b4479-829a-4ac7-8756-52988b12386d" containerName="istio-proxy" Oct 30 19:15:19.115242 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.115223 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.165397 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.164241 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27"] Oct 30 19:15:19.170517 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.170482 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.187160 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.187125 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bff1115-e48c-4282-868e-4a572b4c12cf-istiod-ca-cert\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.187314 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.187175 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-envoy\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.187314 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.187219 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-workload-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.187314 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.187257 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-token\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.187314 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.187278 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-podinfo\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.187493 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.187412 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-credential-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.187493 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.187484 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-workload-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.187562 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.187526 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istiod-ca-cert\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.187632 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.187616 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-envoy\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.187669 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.187646 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-podinfo\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.187712 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.187697 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6xdq\" (UniqueName: \"kubernetes.io/projected/0bff1115-e48c-4282-868e-4a572b4c12cf-kube-api-access-p6xdq\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.187774 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.187760 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-workload-certs\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.187815 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.187797 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-data\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.187852 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.187824 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-credential-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.187888 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.187861 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbpx9\" (UniqueName: \"kubernetes.io/projected/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-kube-api-access-jbpx9\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.187934 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.187918 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-token\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.188014 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.188000 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-data\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.188050 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.188027 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-workload-certs\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.208688 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.208662 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-ae6a138d-openshift-default-dockercfg-hrsvc\"" Oct 30 19:15:19.213565 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.213536 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d"] Oct 30 19:15:19.224735 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.224710 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27"] Oct 30 19:15:19.289753 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.289723 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-data\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.289968 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.289757 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-credential-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.289968 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.289784 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbpx9\" (UniqueName: \"kubernetes.io/projected/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-kube-api-access-jbpx9\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.289968 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.289827 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-token\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.289968 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.289894 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-data\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.289968 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.289918 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-workload-certs\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.290284 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.289970 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bff1115-e48c-4282-868e-4a572b4c12cf-istiod-ca-cert\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.290284 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.290003 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-envoy\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.290284 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.290054 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-workload-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.290284 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.290099 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-token\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.290284 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.290123 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-podinfo\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.290284 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.290156 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-credential-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.290284 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.290187 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-workload-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.290284 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.290227 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istiod-ca-cert\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.290284 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.290256 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-credential-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.290284 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.290264 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-data\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.290885 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.290300 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-data\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.290885 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.290330 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-envoy\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.290885 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.290348 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-workload-certs\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.290885 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.290363 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-podinfo\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.290885 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.290438 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6xdq\" (UniqueName: \"kubernetes.io/projected/0bff1115-e48c-4282-868e-4a572b4c12cf-kube-api-access-p6xdq\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.290885 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.290510 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-workload-certs\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.290885 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.290615 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-workload-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.290885 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.290754 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bff1115-e48c-4282-868e-4a572b4c12cf-istiod-ca-cert\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.290885 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.290773 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-workload-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.291242 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.290913 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-workload-certs\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.291242 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.291129 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istiod-ca-cert\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.291350 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.291237 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-credential-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.292737 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.292712 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-envoy\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.292936 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.292917 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-envoy\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.293343 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.293324 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-podinfo\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.293498 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.293483 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-podinfo\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.347778 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.347739 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbpx9\" (UniqueName: \"kubernetes.io/projected/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-kube-api-access-jbpx9\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.348114 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.348082 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-token\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.349280 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.348121 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6xdq\" (UniqueName: \"kubernetes.io/projected/0bff1115-e48c-4282-868e-4a572b4c12cf-kube-api-access-p6xdq\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.350866 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.350832 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-token\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.429874 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.429837 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:19.450912 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.450876 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:19.510595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.498828 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:19.630164 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.630081 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:19.937839 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.937807 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d"] Oct 30 19:15:19.976823 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:15:19.976793 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1d0e8a5_75ba_4e8d_9168_384428b30c4c.slice/crio-cbe49f966692926c5667e6c778905b2ed4da840d61bce6f25002cf3e40086a2b WatchSource:0}: Error finding container cbe49f966692926c5667e6c778905b2ed4da840d61bce6f25002cf3e40086a2b: Status 404 returned error can't find the container with id cbe49f966692926c5667e6c778905b2ed4da840d61bce6f25002cf3e40086a2b Oct 30 19:15:19.979612 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.979562 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:15:19.979712 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.979648 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:15:19.979712 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.979699 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:15:19.991203 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.991175 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27"] Oct 30 19:15:19.992724 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:15:19.992698 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bff1115_e48c_4282_868e_4a572b4c12cf.slice/crio-11b1066b7c7dc66b4ecdbc62a9a98c6c0c7a3b70103bba72559af537b59aeb94 WatchSource:0}: Error finding container 11b1066b7c7dc66b4ecdbc62a9a98c6c0c7a3b70103bba72559af537b59aeb94: Status 404 returned error can't find the container with id 11b1066b7c7dc66b4ecdbc62a9a98c6c0c7a3b70103bba72559af537b59aeb94 Oct 30 19:15:19.995244 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.995211 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:15:19.995341 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.995288 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:15:19.995341 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:19.995331 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:15:20.142711 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:15:20.142673 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:15:20.646382 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:20.646345 5894 scope.go:122] "RemoveContainer" containerID="52e69ac7b5a1100ccfa68f09f1ec640db627d5d29ec3d493d4d0c80b4faf9c13" Oct 30 19:15:20.819234 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:20.819190 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" event={"ID":"a1d0e8a5-75ba-4e8d-9168-384428b30c4c","Type":"ContainerStarted","Data":"5df39f9c3205bd39046421c940f23a02837a64781b9c56d8efc7e827617eacbf"} Oct 30 19:15:20.819533 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:20.819507 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" event={"ID":"a1d0e8a5-75ba-4e8d-9168-384428b30c4c","Type":"ContainerStarted","Data":"cbe49f966692926c5667e6c778905b2ed4da840d61bce6f25002cf3e40086a2b"} Oct 30 19:15:20.830222 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:20.828524 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" event={"ID":"0bff1115-e48c-4282-868e-4a572b4c12cf","Type":"ContainerStarted","Data":"d9963cdea518f4510e4e475f270bbab446724f750cc6c9e65e9ec19f58b81f70"} Oct 30 19:15:20.830222 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:20.828604 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" event={"ID":"0bff1115-e48c-4282-868e-4a572b4c12cf","Type":"ContainerStarted","Data":"11b1066b7c7dc66b4ecdbc62a9a98c6c0c7a3b70103bba72559af537b59aeb94"} Oct 30 19:15:20.950827 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:20.950718 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" podStartSLOduration=1.9506999820000002 podStartE2EDuration="1.950699982s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:20.948705855 +0000 UTC m=+2191.438833675" watchObservedRunningTime="2026-10-30 19:15:20.950699982 +0000 UTC m=+2191.440827804" Oct 30 19:15:21.087459 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:21.087395 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" podStartSLOduration=2.087374594 podStartE2EDuration="2.087374594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:21.073131618 +0000 UTC m=+2191.563259439" watchObservedRunningTime="2026-10-30 19:15:21.087374594 +0000 UTC m=+2191.577502416" Oct 30 19:15:21.433494 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:21.433372 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:21.450792 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:21.450741 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body= Oct 30 19:15:21.451113 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:21.451071 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" podUID="a1d0e8a5-75ba-4e8d-9168-384428b30c4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" Oct 30 19:15:21.500226 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:21.500194 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:21.501776 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:21.501755 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" start-of-body= Oct 30 19:15:21.501880 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:21.501810 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" podUID="0bff1115-e48c-4282-868e-4a572b4c12cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" Oct 30 19:15:21.890465 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:21.890436 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" event={"ID":"3c1a3f52-3da7-417c-a4e1-a443204576d5","Type":"ContainerStarted","Data":"d65fc5fc929947e8f25f54350f585c9137a02ca7e17fb3f0a03fbf033bf4db76"} Oct 30 19:15:21.890763 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:21.890748 5894 scope.go:122] "RemoveContainer" containerID="d65fc5fc929947e8f25f54350f585c9137a02ca7e17fb3f0a03fbf033bf4db76" Oct 30 19:15:21.890961 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:15:21.890944 5894 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn_odh-ai-gateway-infra(3c1a3f52-3da7-417c-a4e1-a443204576d5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" podUID="3c1a3f52-3da7-417c-a4e1-a443204576d5" Oct 30 19:15:22.451598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:22.451012 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body= Oct 30 19:15:22.451598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:22.451074 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" podUID="a1d0e8a5-75ba-4e8d-9168-384428b30c4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" Oct 30 19:15:22.499979 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:22.499937 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" start-of-body= Oct 30 19:15:22.500148 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:22.500015 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" podUID="0bff1115-e48c-4282-868e-4a572b4c12cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" Oct 30 19:15:22.650600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:22.644291 5894 scope.go:122] "RemoveContainer" containerID="31e81901c69da8f7a3950173418aca80ebc7babd0b80b01d6321a97bca771123" Oct 30 19:15:22.901318 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:22.901282 5894 generic.go:356] "Generic (PLEG): container finished" podID="3c1a3f52-3da7-417c-a4e1-a443204576d5" containerID="d65fc5fc929947e8f25f54350f585c9137a02ca7e17fb3f0a03fbf033bf4db76" exitCode=60 Oct 30 19:15:22.901602 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:22.901545 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" event={"ID":"3c1a3f52-3da7-417c-a4e1-a443204576d5","Type":"ContainerDied","Data":"d65fc5fc929947e8f25f54350f585c9137a02ca7e17fb3f0a03fbf033bf4db76"} Oct 30 19:15:22.901717 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:22.901640 5894 scope.go:122] "RemoveContainer" containerID="52e69ac7b5a1100ccfa68f09f1ec640db627d5d29ec3d493d4d0c80b4faf9c13" Oct 30 19:15:22.902060 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:22.902039 5894 scope.go:122] "RemoveContainer" containerID="d65fc5fc929947e8f25f54350f585c9137a02ca7e17fb3f0a03fbf033bf4db76" Oct 30 19:15:22.902330 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:15:22.902303 5894 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn_odh-ai-gateway-infra(3c1a3f52-3da7-417c-a4e1-a443204576d5)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" podUID="3c1a3f52-3da7-417c-a4e1-a443204576d5" Oct 30 19:15:23.081037 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:23.077781 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn"] Oct 30 19:15:23.225560 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:23.225525 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:23.435828 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:23.435796 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body= Oct 30 19:15:23.439675 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:23.439633 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" podUID="a1d0e8a5-75ba-4e8d-9168-384428b30c4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" Oct 30 19:15:23.500622 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:23.499518 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" start-of-body= Oct 30 19:15:23.500622 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:23.499584 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" podUID="0bff1115-e48c-4282-868e-4a572b4c12cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" Oct 30 19:15:23.660608 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:23.652132 5894 scope.go:122] "RemoveContainer" containerID="d18487a5ce4319aa04df4c10ba4b491c6412bbcf8010af1356942ebc907f799c" Oct 30 19:15:23.660608 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:23.655315 5894 scope.go:122] "RemoveContainer" containerID="ee6c0c024c1210575159545e7ba428f37d45241cd95a2e4294515419928db202" Oct 30 19:15:23.911759 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:23.911721 5894 generic.go:356] "Generic (PLEG): container finished" podID="290f6845-8233-4640-80a5-d25c79c76339" containerID="ce508d9962035bd1835a740a6afe2fc3a3b31f44d9c7535229d990a1bda59299" exitCode=60 Oct 30 19:15:23.911894 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:23.911777 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" event={"ID":"290f6845-8233-4640-80a5-d25c79c76339","Type":"ContainerDied","Data":"ce508d9962035bd1835a740a6afe2fc3a3b31f44d9c7535229d990a1bda59299"} Oct 30 19:15:23.911964 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:23.911914 5894 scope.go:122] "RemoveContainer" containerID="31e81901c69da8f7a3950173418aca80ebc7babd0b80b01d6321a97bca771123" Oct 30 19:15:23.912375 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:23.912354 5894 scope.go:122] "RemoveContainer" containerID="ce508d9962035bd1835a740a6afe2fc3a3b31f44d9c7535229d990a1bda59299" Oct 30 19:15:23.912759 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:15:23.912730 5894 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph_odh-ai-gateway-infra(290f6845-8233-4640-80a5-d25c79c76339)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" podUID="290f6845-8233-4640-80a5-d25c79c76339" Oct 30 19:15:24.386612 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:24.380292 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:24.398878 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:24.398858 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" Oct 30 19:15:24.420835 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:24.420717 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3c1a3f52-3da7-417c-a4e1-a443204576d5-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c1a3f52-3da7-417c-a4e1-a443204576d5-maas-api-service-ca\") pod \"3c1a3f52-3da7-417c-a4e1-a443204576d5\" (UID: \"3c1a3f52-3da7-417c-a4e1-a443204576d5\") " Oct 30 19:15:24.420835 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:24.420777 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c1a3f52-3da7-417c-a4e1-a443204576d5-kube-api-access-d6wjz\" (UniqueName: \"kubernetes.io/projected/3c1a3f52-3da7-417c-a4e1-a443204576d5-kube-api-access-d6wjz\") pod \"3c1a3f52-3da7-417c-a4e1-a443204576d5\" (UID: \"3c1a3f52-3da7-417c-a4e1-a443204576d5\") " Oct 30 19:15:24.421058 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:24.421037 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3c1a3f52-3da7-417c-a4e1-a443204576d5-maas-api-service-ca" pod "3c1a3f52-3da7-417c-a4e1-a443204576d5" (UID: "3c1a3f52-3da7-417c-a4e1-a443204576d5"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:15:24.421554 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:24.421537 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3c1a3f52-3da7-417c-a4e1-a443204576d5-maas-api-service-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:24.422950 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:24.422928 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c1a3f52-3da7-417c-a4e1-a443204576d5-kube-api-access-d6wjz" pod "3c1a3f52-3da7-417c-a4e1-a443204576d5" (UID: "3c1a3f52-3da7-417c-a4e1-a443204576d5"). InnerVolumeSpecName "kube-api-access-d6wjz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:15:24.477631 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:24.440161 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body= Oct 30 19:15:24.477631 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:24.440213 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" podUID="a1d0e8a5-75ba-4e8d-9168-384428b30c4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" Oct 30 19:15:24.500096 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:24.500063 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" start-of-body= Oct 30 19:15:24.500252 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:24.500111 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" podUID="0bff1115-e48c-4282-868e-4a572b4c12cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" Oct 30 19:15:24.522324 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:24.522296 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d6wjz\" (UniqueName: \"kubernetes.io/projected/3c1a3f52-3da7-417c-a4e1-a443204576d5-kube-api-access-d6wjz\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:24.923768 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:24.923734 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" event={"ID":"3ff1c8e5-4048-4daa-88df-5287e0e456c6","Type":"ContainerStarted","Data":"55caf53ae6ccac581b14b6db964c740f2b0b5bf8522d58d23e6a1a030ef7a327"} Oct 30 19:15:24.925920 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:24.925892 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" event={"ID":"416b873d-e218-44c4-8b1d-748d2329ae43","Type":"ContainerStarted","Data":"c0c7867d3ec3f109b29da570d8f0ee4d3024a31d6ee1eccb6398cd47b5cbbdbb"} Oct 30 19:15:24.949598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:24.947283 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" event={"ID":"3c1a3f52-3da7-417c-a4e1-a443204576d5","Type":"ContainerDied","Data":"bf6f09bdb37b3e2e4798e208cc69abe9a576e8aabb23ac259069b3f3c2d1cfae"} Oct 30 19:15:24.949598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:24.947351 5894 scope.go:122] "RemoveContainer" containerID="d65fc5fc929947e8f25f54350f585c9137a02ca7e17fb3f0a03fbf033bf4db76" Oct 30 19:15:24.949598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:24.947448 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn" Oct 30 19:15:25.185365 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.185326 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph"] Oct 30 19:15:25.284082 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.283987 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn"] Oct 30 19:15:25.286740 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.286699 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w0-f5763a-29889795-l68rn"] Oct 30 19:15:25.460381 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.460327 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body= Oct 30 19:15:25.460740 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.460625 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" podUID="a1d0e8a5-75ba-4e8d-9168-384428b30c4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" Oct 30 19:15:25.499760 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.499732 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" start-of-body= Oct 30 19:15:25.499873 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.499784 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" podUID="0bff1115-e48c-4282-868e-4a572b4c12cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" Oct 30 19:15:25.515031 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.515007 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" Oct 30 19:15:25.589202 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.589110 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/290f6845-8233-4640-80a5-d25c79c76339-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/290f6845-8233-4640-80a5-d25c79c76339-maas-api-service-ca\") pod \"290f6845-8233-4640-80a5-d25c79c76339\" (UID: \"290f6845-8233-4640-80a5-d25c79c76339\") " Oct 30 19:15:25.589721 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.589257 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/290f6845-8233-4640-80a5-d25c79c76339-kube-api-access-x94z2\" (UniqueName: \"kubernetes.io/projected/290f6845-8233-4640-80a5-d25c79c76339-kube-api-access-x94z2\") pod \"290f6845-8233-4640-80a5-d25c79c76339\" (UID: \"290f6845-8233-4640-80a5-d25c79c76339\") " Oct 30 19:15:25.589721 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.589451 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/290f6845-8233-4640-80a5-d25c79c76339-maas-api-service-ca" pod "290f6845-8233-4640-80a5-d25c79c76339" (UID: "290f6845-8233-4640-80a5-d25c79c76339"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:15:25.589842 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.589755 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/290f6845-8233-4640-80a5-d25c79c76339-maas-api-service-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:25.591746 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.591721 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/290f6845-8233-4640-80a5-d25c79c76339-kube-api-access-x94z2" pod "290f6845-8233-4640-80a5-d25c79c76339" (UID: "290f6845-8233-4640-80a5-d25c79c76339"). InnerVolumeSpecName "kube-api-access-x94z2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:15:25.694456 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.680487 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c1a3f52-3da7-417c-a4e1-a443204576d5" path="/var/lib/kubelet/pods/3c1a3f52-3da7-417c-a4e1-a443204576d5/volumes" Oct 30 19:15:25.694456 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.690863 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x94z2\" (UniqueName: \"kubernetes.io/projected/290f6845-8233-4640-80a5-d25c79c76339-kube-api-access-x94z2\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:25.982454 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.981813 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" event={"ID":"290f6845-8233-4640-80a5-d25c79c76339","Type":"ContainerDied","Data":"ef36e7836bdf4cb9b0ae945030426b16a06cb38a331a408c2545918cd297d98f"} Oct 30 19:15:25.982454 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.981920 5894 scope.go:122] "RemoveContainer" containerID="ce508d9962035bd1835a740a6afe2fc3a3b31f44d9c7535229d990a1bda59299" Oct 30 19:15:25.982454 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.982045 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph" Oct 30 19:15:25.984996 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.984969 5894 generic.go:356] "Generic (PLEG): container finished" podID="3ff1c8e5-4048-4daa-88df-5287e0e456c6" containerID="55caf53ae6ccac581b14b6db964c740f2b0b5bf8522d58d23e6a1a030ef7a327" exitCode=60 Oct 30 19:15:25.985133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.985115 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" event={"ID":"3ff1c8e5-4048-4daa-88df-5287e0e456c6","Type":"ContainerDied","Data":"55caf53ae6ccac581b14b6db964c740f2b0b5bf8522d58d23e6a1a030ef7a327"} Oct 30 19:15:25.985630 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.985565 5894 scope.go:122] "RemoveContainer" containerID="55caf53ae6ccac581b14b6db964c740f2b0b5bf8522d58d23e6a1a030ef7a327" Oct 30 19:15:25.985969 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:15:25.985944 5894 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s_odh-ai-gateway-infra(3ff1c8e5-4048-4daa-88df-5287e0e456c6)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" podUID="3ff1c8e5-4048-4daa-88df-5287e0e456c6" Oct 30 19:15:25.989612 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.988097 5894 generic.go:356] "Generic (PLEG): container finished" podID="416b873d-e218-44c4-8b1d-748d2329ae43" containerID="c0c7867d3ec3f109b29da570d8f0ee4d3024a31d6ee1eccb6398cd47b5cbbdbb" exitCode=60 Oct 30 19:15:25.989612 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.988202 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" event={"ID":"416b873d-e218-44c4-8b1d-748d2329ae43","Type":"ContainerDied","Data":"c0c7867d3ec3f109b29da570d8f0ee4d3024a31d6ee1eccb6398cd47b5cbbdbb"} Oct 30 19:15:25.989612 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.989228 5894 scope.go:122] "RemoveContainer" containerID="c0c7867d3ec3f109b29da570d8f0ee4d3024a31d6ee1eccb6398cd47b5cbbdbb" Oct 30 19:15:25.989612 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:15:25.989554 5894 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d_odh-ai-gateway-infra(416b873d-e218-44c4-8b1d-748d2329ae43)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" podUID="416b873d-e218-44c4-8b1d-748d2329ae43" Oct 30 19:15:25.993412 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:25.993396 5894 scope.go:122] "RemoveContainer" containerID="ee6c0c024c1210575159545e7ba428f37d45241cd95a2e4294515419928db202" Oct 30 19:15:26.007317 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:26.007297 5894 scope.go:122] "RemoveContainer" containerID="d18487a5ce4319aa04df4c10ba4b491c6412bbcf8010af1356942ebc907f799c" Oct 30 19:15:26.082335 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:26.082302 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s"] Oct 30 19:15:26.183038 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:26.183002 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d"] Oct 30 19:15:26.199289 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:26.199245 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph"] Oct 30 19:15:26.205511 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:26.205480 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w6-41db7c-29889795-p69ph"] Oct 30 19:15:26.447597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:26.435706 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" start-of-body= Oct 30 19:15:26.447597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:26.435763 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" podUID="a1d0e8a5-75ba-4e8d-9168-384428b30c4c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.122:15021/healthz/ready\": dial tcp 10.132.0.122:15021: connect: connection refused" Oct 30 19:15:26.500307 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:26.500058 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" start-of-body= Oct 30 19:15:26.500307 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:26.500104 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" podUID="0bff1115-e48c-4282-868e-4a572b4c12cf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.123:15021/healthz/ready\": dial tcp 10.132.0.123:15021: connect: connection refused" Oct 30 19:15:27.168563 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.168487 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" Oct 30 19:15:27.206401 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.206375 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" Oct 30 19:15:27.214562 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.214539 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/416b873d-e218-44c4-8b1d-748d2329ae43-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/416b873d-e218-44c4-8b1d-748d2329ae43-maas-api-service-ca\") pod \"416b873d-e218-44c4-8b1d-748d2329ae43\" (UID: \"416b873d-e218-44c4-8b1d-748d2329ae43\") " Oct 30 19:15:27.214737 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.214654 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/416b873d-e218-44c4-8b1d-748d2329ae43-kube-api-access-rzl7n\" (UniqueName: \"kubernetes.io/projected/416b873d-e218-44c4-8b1d-748d2329ae43-kube-api-access-rzl7n\") pod \"416b873d-e218-44c4-8b1d-748d2329ae43\" (UID: \"416b873d-e218-44c4-8b1d-748d2329ae43\") " Oct 30 19:15:27.214911 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.214887 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/416b873d-e218-44c4-8b1d-748d2329ae43-maas-api-service-ca" pod "416b873d-e218-44c4-8b1d-748d2329ae43" (UID: "416b873d-e218-44c4-8b1d-748d2329ae43"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:15:27.215200 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.215181 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/416b873d-e218-44c4-8b1d-748d2329ae43-maas-api-service-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:27.217018 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.216995 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/416b873d-e218-44c4-8b1d-748d2329ae43-kube-api-access-rzl7n" pod "416b873d-e218-44c4-8b1d-748d2329ae43" (UID: "416b873d-e218-44c4-8b1d-748d2329ae43"). InnerVolumeSpecName "kube-api-access-rzl7n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:15:27.316548 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.316466 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3ff1c8e5-4048-4daa-88df-5287e0e456c6-kube-api-access-nb45z\" (UniqueName: \"kubernetes.io/projected/3ff1c8e5-4048-4daa-88df-5287e0e456c6-kube-api-access-nb45z\") pod \"3ff1c8e5-4048-4daa-88df-5287e0e456c6\" (UID: \"3ff1c8e5-4048-4daa-88df-5287e0e456c6\") " Oct 30 19:15:27.316806 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.316778 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3ff1c8e5-4048-4daa-88df-5287e0e456c6-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ff1c8e5-4048-4daa-88df-5287e0e456c6-maas-api-service-ca\") pod \"3ff1c8e5-4048-4daa-88df-5287e0e456c6\" (UID: \"3ff1c8e5-4048-4daa-88df-5287e0e456c6\") " Oct 30 19:15:27.317105 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.317084 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ff1c8e5-4048-4daa-88df-5287e0e456c6-maas-api-service-ca" pod "3ff1c8e5-4048-4daa-88df-5287e0e456c6" (UID: "3ff1c8e5-4048-4daa-88df-5287e0e456c6"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:15:27.317314 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.317297 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rzl7n\" (UniqueName: \"kubernetes.io/projected/416b873d-e218-44c4-8b1d-748d2329ae43-kube-api-access-rzl7n\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:27.317359 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.317317 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3ff1c8e5-4048-4daa-88df-5287e0e456c6-maas-api-service-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:27.318606 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.318583 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ff1c8e5-4048-4daa-88df-5287e0e456c6-kube-api-access-nb45z" pod "3ff1c8e5-4048-4daa-88df-5287e0e456c6" (UID: "3ff1c8e5-4048-4daa-88df-5287e0e456c6"). InnerVolumeSpecName "kube-api-access-nb45z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:15:27.418428 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.418394 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nb45z\" (UniqueName: \"kubernetes.io/projected/3ff1c8e5-4048-4daa-88df-5287e0e456c6-kube-api-access-nb45z\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:15:27.435519 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.435492 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:27.435820 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.435801 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:27.436426 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.436404 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:15:27.505149 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.505119 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:27.506469 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.506145 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:27.506942 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.506922 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:15:27.651291 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:27.651260 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="290f6845-8233-4640-80a5-d25c79c76339" path="/var/lib/kubelet/pods/290f6845-8233-4640-80a5-d25c79c76339/volumes" Oct 30 19:15:28.010278 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:28.010194 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" Oct 30 19:15:28.010278 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:28.010219 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s" event={"ID":"3ff1c8e5-4048-4daa-88df-5287e0e456c6","Type":"ContainerDied","Data":"315fbb343bf7407a07bb1139707dde97e506dbda1a5e104ab09f9a9080287d1b"} Oct 30 19:15:28.010511 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:28.010282 5894 scope.go:122] "RemoveContainer" containerID="55caf53ae6ccac581b14b6db964c740f2b0b5bf8522d58d23e6a1a030ef7a327" Oct 30 19:15:28.012428 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:28.012406 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" Oct 30 19:15:28.012875 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:28.012845 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d" event={"ID":"416b873d-e218-44c4-8b1d-748d2329ae43","Type":"ContainerDied","Data":"02ed9951642a421f2c2166852390ffd3a909f8f249c9573484cbf699b291c814"} Oct 30 19:15:28.020824 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:28.020808 5894 scope.go:122] "RemoveContainer" containerID="c0c7867d3ec3f109b29da570d8f0ee4d3024a31d6ee1eccb6398cd47b5cbbdbb" Oct 30 19:15:28.176372 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:28.176334 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s"] Oct 30 19:15:28.180921 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:28.180897 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w2-66bdc0-29889795-zjx8s"] Oct 30 19:15:28.307889 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:28.307681 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d"] Oct 30 19:15:28.330855 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:28.328230 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:28.346595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:28.343232 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-api-b-903a82-29889795-wsj7d"] Oct 30 19:15:29.653773 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:29.653737 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3ff1c8e5-4048-4daa-88df-5287e0e456c6" path="/var/lib/kubelet/pods/3ff1c8e5-4048-4daa-88df-5287e0e456c6/volumes" Oct 30 19:15:29.654214 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:29.654199 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="416b873d-e218-44c4-8b1d-748d2329ae43" path="/var/lib/kubelet/pods/416b873d-e218-44c4-8b1d-748d2329ae43/volumes" Oct 30 19:15:33.008109 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:33.008072 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:34.251924 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.251883 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5"] Oct 30 19:15:34.252696 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252675 5894 cpu_manager.go:414] "containerMap: removing container" podUID="3c1a3f52-3da7-417c-a4e1-a443204576d5" containerName="cleanup" Oct 30 19:15:34.252696 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252696 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c1a3f52-3da7-417c-a4e1-a443204576d5" containerName="cleanup" Oct 30 19:15:34.252863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252716 5894 cpu_manager.go:414] "containerMap: removing container" podUID="290f6845-8233-4640-80a5-d25c79c76339" containerName="cleanup" Oct 30 19:15:34.252863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252724 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="290f6845-8233-4640-80a5-d25c79c76339" containerName="cleanup" Oct 30 19:15:34.252863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252736 5894 cpu_manager.go:414] "containerMap: removing container" podUID="416b873d-e218-44c4-8b1d-748d2329ae43" containerName="cleanup" Oct 30 19:15:34.252863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252743 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="416b873d-e218-44c4-8b1d-748d2329ae43" containerName="cleanup" Oct 30 19:15:34.252863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252751 5894 cpu_manager.go:414] "containerMap: removing container" podUID="290f6845-8233-4640-80a5-d25c79c76339" containerName="cleanup" Oct 30 19:15:34.252863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252758 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="290f6845-8233-4640-80a5-d25c79c76339" containerName="cleanup" Oct 30 19:15:34.252863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252789 5894 cpu_manager.go:414] "containerMap: removing container" podUID="416b873d-e218-44c4-8b1d-748d2329ae43" containerName="cleanup" Oct 30 19:15:34.252863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252797 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="416b873d-e218-44c4-8b1d-748d2329ae43" containerName="cleanup" Oct 30 19:15:34.252863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252809 5894 cpu_manager.go:414] "containerMap: removing container" podUID="3ff1c8e5-4048-4daa-88df-5287e0e456c6" containerName="cleanup" Oct 30 19:15:34.252863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252815 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ff1c8e5-4048-4daa-88df-5287e0e456c6" containerName="cleanup" Oct 30 19:15:34.252863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252831 5894 cpu_manager.go:414] "containerMap: removing container" podUID="3ff1c8e5-4048-4daa-88df-5287e0e456c6" containerName="cleanup" Oct 30 19:15:34.252863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252838 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ff1c8e5-4048-4daa-88df-5287e0e456c6" containerName="cleanup" Oct 30 19:15:34.252863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252859 5894 cpu_manager.go:414] "containerMap: removing container" podUID="290f6845-8233-4640-80a5-d25c79c76339" containerName="cleanup" Oct 30 19:15:34.252863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252866 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="290f6845-8233-4640-80a5-d25c79c76339" containerName="cleanup" Oct 30 19:15:34.253620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252899 5894 cpu_manager.go:414] "containerMap: removing container" podUID="3ff1c8e5-4048-4daa-88df-5287e0e456c6" containerName="cleanup" Oct 30 19:15:34.253620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252906 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3ff1c8e5-4048-4daa-88df-5287e0e456c6" containerName="cleanup" Oct 30 19:15:34.253620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252933 5894 cpu_manager.go:414] "containerMap: removing container" podUID="3c1a3f52-3da7-417c-a4e1-a443204576d5" containerName="cleanup" Oct 30 19:15:34.253620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.252940 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c1a3f52-3da7-417c-a4e1-a443204576d5" containerName="cleanup" Oct 30 19:15:34.253620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.253092 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="416b873d-e218-44c4-8b1d-748d2329ae43" containerName="cleanup" Oct 30 19:15:34.253620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.253105 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ff1c8e5-4048-4daa-88df-5287e0e456c6" containerName="cleanup" Oct 30 19:15:34.253620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.253115 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ff1c8e5-4048-4daa-88df-5287e0e456c6" containerName="cleanup" Oct 30 19:15:34.253620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.253127 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="290f6845-8233-4640-80a5-d25c79c76339" containerName="cleanup" Oct 30 19:15:34.253620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.253140 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c1a3f52-3da7-417c-a4e1-a443204576d5" containerName="cleanup" Oct 30 19:15:34.253620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.253148 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="416b873d-e218-44c4-8b1d-748d2329ae43" containerName="cleanup" Oct 30 19:15:34.253620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.253160 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c1a3f52-3da7-417c-a4e1-a443204576d5" containerName="cleanup" Oct 30 19:15:34.253620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.253168 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c1a3f52-3da7-417c-a4e1-a443204576d5" containerName="cleanup" Oct 30 19:15:34.253620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.253179 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="290f6845-8233-4640-80a5-d25c79c76339" containerName="cleanup" Oct 30 19:15:34.253620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.253187 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="416b873d-e218-44c4-8b1d-748d2329ae43" containerName="cleanup" Oct 30 19:15:34.253620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.253196 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="290f6845-8233-4640-80a5-d25c79c76339" containerName="cleanup" Oct 30 19:15:34.258429 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.258398 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.325161 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.325115 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-workload-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.325324 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.325226 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-token\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.325402 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.325353 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrmql\" (UniqueName: \"kubernetes.io/projected/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-kube-api-access-nrmql\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.325402 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.325384 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-envoy\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.325519 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.325481 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-podinfo\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.325621 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.325596 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-credential-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.325711 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.325691 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-data\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.325772 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.325729 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-workload-certs\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.325816 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.325776 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istiod-ca-cert\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.427296 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.427258 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-data\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.427296 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.427300 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-workload-certs\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.427509 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.427329 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istiod-ca-cert\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.427509 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.427362 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-workload-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.427509 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.427395 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-token\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.427509 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.427497 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrmql\" (UniqueName: \"kubernetes.io/projected/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-kube-api-access-nrmql\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.427716 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.427518 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-envoy\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.427716 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.427537 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-podinfo\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.427716 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.427630 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-credential-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.427868 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.427746 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-workload-certs\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.427868 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.427767 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-data\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.427981 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.427863 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-workload-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.427981 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.427966 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-credential-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.428250 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.428225 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istiod-ca-cert\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.429824 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.429803 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-envoy\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.430087 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.430069 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-podinfo\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.460073 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.460038 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5"] Oct 30 19:15:34.485090 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.485063 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-token\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.485313 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.485285 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrmql\" (UniqueName: \"kubernetes.io/projected/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-kube-api-access-nrmql\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:34.572518 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:34.572430 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:35.012453 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:35.012427 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5"] Oct 30 19:15:35.013242 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:15:35.013217 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d4f4ad1_2be4_4c2a_8614_1b71cd7365da.slice/crio-651a519cf6b5f13afb4c6e532350242fd87ce09c50bf027165c7b9c9af404ada WatchSource:0}: Error finding container 651a519cf6b5f13afb4c6e532350242fd87ce09c50bf027165c7b9c9af404ada: Status 404 returned error can't find the container with id 651a519cf6b5f13afb4c6e532350242fd87ce09c50bf027165c7b9c9af404ada Oct 30 19:15:35.016216 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:35.016184 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:15:35.016317 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:35.016275 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:15:35.016378 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:35.016359 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:15:35.069541 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:35.069514 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" event={"ID":"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da","Type":"ContainerStarted","Data":"651a519cf6b5f13afb4c6e532350242fd87ce09c50bf027165c7b9c9af404ada"} Oct 30 19:15:36.078128 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:36.078091 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" event={"ID":"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da","Type":"ContainerStarted","Data":"551910f8f6104ea23e3ee182d2863d1fd37b505e613e49aa1f8216fc8da79a07"} Oct 30 19:15:36.252895 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:36.252836 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" podStartSLOduration=2.252820339 podStartE2EDuration="2.252820339s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:36.248013549 +0000 UTC m=+2206.738141372" watchObservedRunningTime="2026-10-30 19:15:36.252820339 +0000 UTC m=+2206.742948229" Oct 30 19:15:36.572797 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:36.572761 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:36.574590 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:36.574550 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body= Oct 30 19:15:36.574714 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:36.574613 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" podUID="1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" Oct 30 19:15:37.572809 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:37.572766 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body= Oct 30 19:15:37.573257 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:37.572826 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" podUID="1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" Oct 30 19:15:37.983231 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:37.979307 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:38.573098 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:38.573061 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body= Oct 30 19:15:38.573491 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:38.573115 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" podUID="1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" Oct 30 19:15:39.573778 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:39.573743 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body= Oct 30 19:15:39.574173 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:39.573809 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" podUID="1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" Oct 30 19:15:40.573101 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:40.573066 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" start-of-body= Oct 30 19:15:40.573274 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:40.573146 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" podUID="1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.124:15021/healthz/ready\": dial tcp 10.132.0.124:15021: connect: connection refused" Oct 30 19:15:41.577731 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:41.577701 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:41.578134 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:41.578114 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:41.578811 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:41.578794 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:15:42.789261 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:42.789224 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:47.504238 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:47.504203 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:49.314034 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.313995 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:49.367052 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.367020 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d"] Oct 30 19:15:49.367706 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.367687 5894 cpu_manager.go:414] "containerMap: removing container" podUID="3c1a3f52-3da7-417c-a4e1-a443204576d5" containerName="cleanup" Oct 30 19:15:49.367755 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.367707 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c1a3f52-3da7-417c-a4e1-a443204576d5" containerName="cleanup" Oct 30 19:15:49.367755 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.367734 5894 cpu_manager.go:414] "containerMap: removing container" podUID="416b873d-e218-44c4-8b1d-748d2329ae43" containerName="cleanup" Oct 30 19:15:49.367755 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.367738 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="416b873d-e218-44c4-8b1d-748d2329ae43" containerName="cleanup" Oct 30 19:15:49.367867 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.367858 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="3ff1c8e5-4048-4daa-88df-5287e0e456c6" containerName="cleanup" Oct 30 19:15:49.371286 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.371265 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.438506 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.438467 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-data\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.438876 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.438852 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-workload-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.439124 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.439104 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-token\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.510653 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.510616 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8jw4c\" (UniqueName: \"kubernetes.io/projected/1a6b1680-232d-4c50-beee-d3967a408fd4-kube-api-access-8jw4c\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.510844 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.510787 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a6b1680-232d-4c50-beee-d3967a408fd4-istiod-ca-cert\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.510914 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.510844 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-envoy\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.510914 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.510882 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-podinfo\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.511014 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.510913 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-workload-certs\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.511072 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.511012 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-credential-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.511308 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.503886 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:49.511308 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.506051 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d"] Oct 30 19:15:49.632652 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.617477 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8jw4c\" (UniqueName: \"kubernetes.io/projected/1a6b1680-232d-4c50-beee-d3967a408fd4-kube-api-access-8jw4c\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.709289 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.709247 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a6b1680-232d-4c50-beee-d3967a408fd4-istiod-ca-cert\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.711079 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.710951 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-envoy\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.711238 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.711151 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-podinfo\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.711238 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.711190 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-workload-certs\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.711365 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.711340 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-credential-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.711422 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.711389 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a6b1680-232d-4c50-beee-d3967a408fd4-istiod-ca-cert\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.711512 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.711492 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-data\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.711652 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.711622 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-workload-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.711735 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.711719 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-workload-certs\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.712246 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.711724 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-token\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.712246 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.712016 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-data\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.712421 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.712241 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-credential-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.712517 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.712498 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-workload-socket\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.715022 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.714995 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8jw4c\" (UniqueName: \"kubernetes.io/projected/1a6b1680-232d-4c50-beee-d3967a408fd4-kube-api-access-8jw4c\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.719423 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.719378 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-podinfo\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.720819 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.720802 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-envoy\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:49.768240 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:49.768200 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-token\") pod \"e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:50.001872 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:50.001841 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:50.140478 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:15:50.140399 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:15:50.230339 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:50.230303 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:50.445842 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:15:50.445810 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a6b1680_232d_4c50_beee_d3967a408fd4.slice/crio-be78832875ca73a21d632da72b6d243b1c38d4918b0dd8336d3f4ff10662bda9 WatchSource:0}: Error finding container be78832875ca73a21d632da72b6d243b1c38d4918b0dd8336d3f4ff10662bda9: Status 404 returned error can't find the container with id be78832875ca73a21d632da72b6d243b1c38d4918b0dd8336d3f4ff10662bda9 Oct 30 19:15:50.446182 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:50.446038 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d"] Oct 30 19:15:50.448780 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:50.448735 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:15:50.448878 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:50.448812 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:15:50.448878 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:50.448841 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:15:51.274767 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:51.274715 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" event={"ID":"1a6b1680-232d-4c50-beee-d3967a408fd4","Type":"ContainerStarted","Data":"03975e70e8168300f82971dfc7ed26bd583c25fd8d0a5b872ba6331498f90188"} Oct 30 19:15:51.275078 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:51.275054 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" event={"ID":"1a6b1680-232d-4c50-beee-d3967a408fd4","Type":"ContainerStarted","Data":"be78832875ca73a21d632da72b6d243b1c38d4918b0dd8336d3f4ff10662bda9"} Oct 30 19:15:51.507814 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:51.507761 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" podStartSLOduration=2.50774819 podStartE2EDuration="2.50774819s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:15:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:15:51.455378129 +0000 UTC m=+2221.945505950" watchObservedRunningTime="2026-10-30 19:15:51.50774819 +0000 UTC m=+2221.997876037" Oct 30 19:15:52.002731 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:52.002698 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:52.004098 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:52.004074 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body= Oct 30 19:15:52.004233 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:52.004115 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" podUID="1a6b1680-232d-4c50-beee-d3967a408fd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" Oct 30 19:15:53.002919 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:53.002823 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body= Oct 30 19:15:53.003976 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:53.003493 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" podUID="1a6b1680-232d-4c50-beee-d3967a408fd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" Oct 30 19:15:53.904857 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:53.904815 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:54.003430 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:54.003396 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body= Oct 30 19:15:54.003837 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:54.003464 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" podUID="1a6b1680-232d-4c50-beee-d3967a408fd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" Oct 30 19:15:54.274021 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:54.273923 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:54.490845 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:54.490806 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:54.907300 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:54.907265 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:55.002778 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:55.002742 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body= Oct 30 19:15:55.002946 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:55.002804 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" podUID="1a6b1680-232d-4c50-beee-d3967a408fd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" Oct 30 19:15:55.888601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:55.887890 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl"] Oct 30 19:15:55.888601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:55.888127 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" podUID="14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" containerName="istio-proxy" containerID="cri-o://41a16f4cd21dccb62e094e2b7b86b0fe430f3ecec20b366ae4d9179ba5755340" gracePeriod=30 Oct 30 19:15:55.991468 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:55.991431 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57"] Oct 30 19:15:55.991948 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:55.991918 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" podUID="bb0794ea-65d0-4050-81eb-2b961d482d33" containerName="istio-proxy" containerID="cri-o://cf1a378a78e8889c87103659cafbc0a2c3bd99865dccd9c6261398862f9d2631" gracePeriod=30 Oct 30 19:15:56.005505 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:56.005471 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body= Oct 30 19:15:56.007906 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:56.007856 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" podUID="1a6b1680-232d-4c50-beee-d3967a408fd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" Oct 30 19:15:56.604030 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:56.603944 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:56.903884 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:56.903844 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:57.002826 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:57.002785 5894 patch_prober.go:29] interesting pod/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" start-of-body= Oct 30 19:15:57.003030 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:57.002847 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" podUID="1a6b1680-232d-4c50-beee-d3967a408fd4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.125:15021/healthz/ready\": dial tcp 10.132.0.125:15021: connect: connection refused" Oct 30 19:15:58.026198 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:58.026161 5894 patch_prober.go:29] interesting pod/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:15:58.026645 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:58.026219 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" podUID="14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:15:58.194405 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:58.194376 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:58.194707 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:58.194685 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:58.195411 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:58.195390 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:15:58.456971 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:58.456936 5894 patch_prober.go:29] interesting pod/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:15:58.457150 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:58.457001 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" podUID="bb0794ea-65d0-4050-81eb-2b961d482d33" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:15:59.525365 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:59.525329 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:15:59.705217 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:15:59.705187 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:00.628438 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:00.628407 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp"] Oct 30 19:16:00.670741 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:00.670703 5894 patch_prober.go:29] interesting pod/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:16:00.670908 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:00.670765 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" podUID="b0479234-1db6-465f-9454-54d777b09538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:16:00.926212 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:00.926121 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:01.132589 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.131809 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:01.292107 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.291735 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:16:01.326491 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.326450 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:01.334669 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.334636 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-workload-socket\") pod \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " Oct 30 19:16:01.334836 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.334695 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-kube-api-access-s7x49\" (UniqueName: \"kubernetes.io/projected/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-kube-api-access-s7x49\") pod \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " Oct 30 19:16:01.334836 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.334758 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-data\") pod \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " Oct 30 19:16:01.334836 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.334801 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-credential-socket\") pod \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " Oct 30 19:16:01.335033 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.334897 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-podinfo\") pod \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " Oct 30 19:16:01.335033 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.334951 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-token\" (UniqueName: \"kubernetes.io/projected/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-token\") pod \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " Oct 30 19:16:01.335033 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.335026 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-envoy\") pod \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " Oct 30 19:16:01.335194 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.335096 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istiod-ca-cert\") pod \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " Oct 30 19:16:01.335194 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.335145 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-data" pod "14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" (UID: "14cda751-f1b9-4b48-9fe2-bcd0a2909d0c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:16:01.335301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.335213 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-workload-certs\") pod \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\" (UID: \"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c\") " Oct 30 19:16:01.335749 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.335435 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-workload-socket" pod "14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" (UID: "14cda751-f1b9-4b48-9fe2-bcd0a2909d0c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:16:01.335749 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.335525 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istiod-ca-cert" pod "14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" (UID: "14cda751-f1b9-4b48-9fe2-bcd0a2909d0c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:16:01.335913 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.335828 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-credential-socket" pod "14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" (UID: "14cda751-f1b9-4b48-9fe2-bcd0a2909d0c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:16:01.336080 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.336035 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-workload-certs" pod "14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" (UID: "14cda751-f1b9-4b48-9fe2-bcd0a2909d0c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:16:01.336181 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.336144 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:01.336181 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.336166 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:01.336312 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.336181 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:01.336312 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.336194 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:01.336312 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.336226 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:01.338321 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.338276 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-envoy" pod "14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" (UID: "14cda751-f1b9-4b48-9fe2-bcd0a2909d0c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:16:01.338962 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.338928 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-token" pod "14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" (UID: "14cda751-f1b9-4b48-9fe2-bcd0a2909d0c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:16:01.339749 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.339728 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-podinfo" pod "14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" (UID: "14cda751-f1b9-4b48-9fe2-bcd0a2909d0c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:16:01.339829 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.339761 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-kube-api-access-s7x49" pod "14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" (UID: "14cda751-f1b9-4b48-9fe2-bcd0a2909d0c"). InnerVolumeSpecName "kube-api-access-s7x49". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:16:01.364113 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.364083 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:16:01.437352 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.437315 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-token\" (UniqueName: \"kubernetes.io/projected/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-token\") pod \"bb0794ea-65d0-4050-81eb-2b961d482d33\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " Oct 30 19:16:01.437542 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.437404 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bb0794ea-65d0-4050-81eb-2b961d482d33-kube-api-access-dnt6m\" (UniqueName: \"kubernetes.io/projected/bb0794ea-65d0-4050-81eb-2b961d482d33-kube-api-access-dnt6m\") pod \"bb0794ea-65d0-4050-81eb-2b961d482d33\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " Oct 30 19:16:01.437542 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.437455 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bb0794ea-65d0-4050-81eb-2b961d482d33-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb0794ea-65d0-4050-81eb-2b961d482d33-istiod-ca-cert\") pod \"bb0794ea-65d0-4050-81eb-2b961d482d33\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " Oct 30 19:16:01.437724 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.437541 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-credential-socket\") pod \"bb0794ea-65d0-4050-81eb-2b961d482d33\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " Oct 30 19:16:01.437724 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.437631 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-data\") pod \"bb0794ea-65d0-4050-81eb-2b961d482d33\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " Oct 30 19:16:01.437997 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.437973 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-data" pod "bb0794ea-65d0-4050-81eb-2b961d482d33" (UID: "bb0794ea-65d0-4050-81eb-2b961d482d33"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:16:01.438103 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.438075 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb0794ea-65d0-4050-81eb-2b961d482d33-istiod-ca-cert" pod "bb0794ea-65d0-4050-81eb-2b961d482d33" (UID: "bb0794ea-65d0-4050-81eb-2b961d482d33"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:16:01.438204 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.438186 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-credential-socket" pod "bb0794ea-65d0-4050-81eb-2b961d482d33" (UID: "bb0794ea-65d0-4050-81eb-2b961d482d33"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:16:01.438341 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.438319 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-podinfo\") pod \"bb0794ea-65d0-4050-81eb-2b961d482d33\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " Oct 30 19:16:01.438442 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.438414 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-envoy\") pod \"bb0794ea-65d0-4050-81eb-2b961d482d33\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " Oct 30 19:16:01.438500 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.438465 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-workload-certs\") pod \"bb0794ea-65d0-4050-81eb-2b961d482d33\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " Oct 30 19:16:01.438668 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.438553 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-workload-socket\") pod \"bb0794ea-65d0-4050-81eb-2b961d482d33\" (UID: \"bb0794ea-65d0-4050-81eb-2b961d482d33\") " Oct 30 19:16:01.439255 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.439227 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:01.439365 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.439258 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:01.439365 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.439275 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:01.439365 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.439290 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7x49\" (UniqueName: \"kubernetes.io/projected/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-kube-api-access-s7x49\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:01.439365 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.439305 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb0794ea-65d0-4050-81eb-2b961d482d33-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:01.439365 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.439318 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:01.439365 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.439331 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:01.440032 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.440004 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-workload-certs" pod "bb0794ea-65d0-4050-81eb-2b961d482d33" (UID: "bb0794ea-65d0-4050-81eb-2b961d482d33"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:16:01.440320 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.440297 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-workload-socket" pod "bb0794ea-65d0-4050-81eb-2b961d482d33" (UID: "bb0794ea-65d0-4050-81eb-2b961d482d33"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:16:01.445183 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.441293 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb0794ea-65d0-4050-81eb-2b961d482d33-kube-api-access-dnt6m" pod "bb0794ea-65d0-4050-81eb-2b961d482d33" (UID: "bb0794ea-65d0-4050-81eb-2b961d482d33"). InnerVolumeSpecName "kube-api-access-dnt6m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:16:01.445183 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.443999 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-podinfo" pod "bb0794ea-65d0-4050-81eb-2b961d482d33" (UID: "bb0794ea-65d0-4050-81eb-2b961d482d33"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:16:01.445183 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.444012 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-token" pod "bb0794ea-65d0-4050-81eb-2b961d482d33" (UID: "bb0794ea-65d0-4050-81eb-2b961d482d33"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:16:01.445183 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.444373 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-envoy" pod "bb0794ea-65d0-4050-81eb-2b961d482d33" (UID: "bb0794ea-65d0-4050-81eb-2b961d482d33"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:16:01.522617 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.522500 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:01.540452 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.540422 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:01.540452 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.540454 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dnt6m\" (UniqueName: \"kubernetes.io/projected/bb0794ea-65d0-4050-81eb-2b961d482d33-kube-api-access-dnt6m\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:01.540683 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.540471 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:01.540683 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.540486 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:01.540683 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.540499 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:01.540683 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.540513 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb0794ea-65d0-4050-81eb-2b961d482d33-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:01.643216 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.643181 5894 generic.go:356] "Generic (PLEG): container finished" podID="14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" containerID="41a16f4cd21dccb62e094e2b7b86b0fe430f3ecec20b366ae4d9179ba5755340" exitCode=0 Oct 30 19:16:01.643726 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.643318 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" Oct 30 19:16:01.650503 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.649725 5894 generic.go:356] "Generic (PLEG): container finished" podID="bb0794ea-65d0-4050-81eb-2b961d482d33" containerID="cf1a378a78e8889c87103659cafbc0a2c3bd99865dccd9c6261398862f9d2631" exitCode=0 Oct 30 19:16:01.650503 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.649874 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" Oct 30 19:16:01.657290 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.657259 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" event={"ID":"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c","Type":"ContainerDied","Data":"41a16f4cd21dccb62e094e2b7b86b0fe430f3ecec20b366ae4d9179ba5755340"} Oct 30 19:16:01.657413 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.657296 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl" event={"ID":"14cda751-f1b9-4b48-9fe2-bcd0a2909d0c","Type":"ContainerDied","Data":"613c89f370a3217d3dec1bb7c3528eee9e1bd0aac8bb018cc7affbe547291362"} Oct 30 19:16:01.657413 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.657311 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" event={"ID":"bb0794ea-65d0-4050-81eb-2b961d482d33","Type":"ContainerDied","Data":"cf1a378a78e8889c87103659cafbc0a2c3bd99865dccd9c6261398862f9d2631"} Oct 30 19:16:01.657413 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.657345 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57" event={"ID":"bb0794ea-65d0-4050-81eb-2b961d482d33","Type":"ContainerDied","Data":"d301c62d4335105e16c7f2c5500bc4891a537d757214b09d35823a2759f9eba6"} Oct 30 19:16:01.657413 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.657375 5894 scope.go:122] "RemoveContainer" containerID="41a16f4cd21dccb62e094e2b7b86b0fe430f3ecec20b366ae4d9179ba5755340" Oct 30 19:16:01.765250 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.765155 5894 scope.go:122] "RemoveContainer" containerID="41a16f4cd21dccb62e094e2b7b86b0fe430f3ecec20b366ae4d9179ba5755340" Oct 30 19:16:01.768086 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:16:01.767921 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"41a16f4cd21dccb62e094e2b7b86b0fe430f3ecec20b366ae4d9179ba5755340\": container with ID starting with 41a16f4cd21dccb62e094e2b7b86b0fe430f3ecec20b366ae4d9179ba5755340 not found: ID does not exist" containerID="41a16f4cd21dccb62e094e2b7b86b0fe430f3ecec20b366ae4d9179ba5755340" Oct 30 19:16:01.768086 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.767969 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"41a16f4cd21dccb62e094e2b7b86b0fe430f3ecec20b366ae4d9179ba5755340"} err="failed to get container status \"41a16f4cd21dccb62e094e2b7b86b0fe430f3ecec20b366ae4d9179ba5755340\": rpc error: code = NotFound desc = could not find container \"41a16f4cd21dccb62e094e2b7b86b0fe430f3ecec20b366ae4d9179ba5755340\": container with ID starting with 41a16f4cd21dccb62e094e2b7b86b0fe430f3ecec20b366ae4d9179ba5755340 not found: ID does not exist" Oct 30 19:16:01.768086 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.767994 5894 scope.go:122] "RemoveContainer" containerID="cf1a378a78e8889c87103659cafbc0a2c3bd99865dccd9c6261398862f9d2631" Oct 30 19:16:01.797402 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.797055 5894 scope.go:122] "RemoveContainer" containerID="cf1a378a78e8889c87103659cafbc0a2c3bd99865dccd9c6261398862f9d2631" Oct 30 19:16:01.797775 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:16:01.797665 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cf1a378a78e8889c87103659cafbc0a2c3bd99865dccd9c6261398862f9d2631\": container with ID starting with cf1a378a78e8889c87103659cafbc0a2c3bd99865dccd9c6261398862f9d2631 not found: ID does not exist" containerID="cf1a378a78e8889c87103659cafbc0a2c3bd99865dccd9c6261398862f9d2631" Oct 30 19:16:01.797775 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.797705 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cf1a378a78e8889c87103659cafbc0a2c3bd99865dccd9c6261398862f9d2631"} err="failed to get container status \"cf1a378a78e8889c87103659cafbc0a2c3bd99865dccd9c6261398862f9d2631\": rpc error: code = NotFound desc = could not find container \"cf1a378a78e8889c87103659cafbc0a2c3bd99865dccd9c6261398862f9d2631\": container with ID starting with cf1a378a78e8889c87103659cafbc0a2c3bd99865dccd9c6261398862f9d2631 not found: ID does not exist" Oct 30 19:16:01.878064 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.878019 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl"] Oct 30 19:16:01.883335 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:01.883298 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-qmkwl"] Oct 30 19:16:02.065646 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:02.064631 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57"] Oct 30 19:16:02.075425 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:02.075386 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-p6l57"] Oct 30 19:16:03.650484 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:03.650444 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" path="/var/lib/kubelet/pods/14cda751-f1b9-4b48-9fe2-bcd0a2909d0c/volumes" Oct 30 19:16:03.651192 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:03.651169 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bb0794ea-65d0-4050-81eb-2b961d482d33" path="/var/lib/kubelet/pods/bb0794ea-65d0-4050-81eb-2b961d482d33/volumes" Oct 30 19:16:03.665902 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:03.665876 5894 generic.go:356] "Generic (PLEG): container finished" podID="b0479234-1db6-465f-9454-54d777b09538" containerID="cdc0d40f50f26e4e40419f58a5a929522a0e258cae6deefd5c7127e887a48cc7" exitCode=0 Oct 30 19:16:03.666033 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:03.665944 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" event={"ID":"b0479234-1db6-465f-9454-54d777b09538","Type":"ContainerDied","Data":"cdc0d40f50f26e4e40419f58a5a929522a0e258cae6deefd5c7127e887a48cc7"} Oct 30 19:16:03.666357 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:03.666342 5894 scope.go:122] "RemoveContainer" containerID="cdc0d40f50f26e4e40419f58a5a929522a0e258cae6deefd5c7127e887a48cc7" Oct 30 19:16:03.667821 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:03.667794 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:16:03.667925 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:03.667875 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:16:03.667989 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:03.667923 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:16:04.570905 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:04.570867 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:04.674711 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:04.674668 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" event={"ID":"b0479234-1db6-465f-9454-54d777b09538","Type":"ContainerStarted","Data":"d0f29a229fd44421812be3865231c2954a10a8d17e421382155f16aa9bd32f78"} Oct 30 19:16:05.193511 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:05.193476 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:16:05.193738 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:05.193680 5894 patch_prober.go:29] interesting pod/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body= Oct 30 19:16:05.193816 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:05.193732 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" podUID="b0479234-1db6-465f-9454-54d777b09538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" Oct 30 19:16:05.643651 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:05.639649 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp" podUID="9d683e67-87f6-4d4b-a208-0ebea816c5a7" containerName="payload-pre-processing" containerID="cri-o://fcc84f87901701cca3cff82cb7b27fa1229e305713851f9d1e94bd8346985db3" gracePeriod=25 Oct 30 19:16:05.995447 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:05.995408 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp" Oct 30 19:16:06.052876 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:06.051206 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9d683e67-87f6-4d4b-a208-0ebea816c5a7-kube-api-access-sl2fs\" (UniqueName: \"kubernetes.io/projected/9d683e67-87f6-4d4b-a208-0ebea816c5a7-kube-api-access-sl2fs\") pod \"9d683e67-87f6-4d4b-a208-0ebea816c5a7\" (UID: \"9d683e67-87f6-4d4b-a208-0ebea816c5a7\") " Oct 30 19:16:06.052876 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:06.051322 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9d683e67-87f6-4d4b-a208-0ebea816c5a7-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d683e67-87f6-4d4b-a208-0ebea816c5a7-plugins-config-volume\") pod \"9d683e67-87f6-4d4b-a208-0ebea816c5a7\" (UID: \"9d683e67-87f6-4d4b-a208-0ebea816c5a7\") " Oct 30 19:16:06.053348 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:06.053315 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d683e67-87f6-4d4b-a208-0ebea816c5a7-plugins-config-volume" pod "9d683e67-87f6-4d4b-a208-0ebea816c5a7" (UID: "9d683e67-87f6-4d4b-a208-0ebea816c5a7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:16:06.053563 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:06.053545 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d683e67-87f6-4d4b-a208-0ebea816c5a7-kube-api-access-sl2fs" pod "9d683e67-87f6-4d4b-a208-0ebea816c5a7" (UID: "9d683e67-87f6-4d4b-a208-0ebea816c5a7"). InnerVolumeSpecName "kube-api-access-sl2fs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:16:06.158395 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:06.158356 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sl2fs\" (UniqueName: \"kubernetes.io/projected/9d683e67-87f6-4d4b-a208-0ebea816c5a7-kube-api-access-sl2fs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:06.158395 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:06.158392 5894 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9d683e67-87f6-4d4b-a208-0ebea816c5a7-plugins-config-volume\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:06.194244 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:06.194214 5894 patch_prober.go:29] interesting pod/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body= Oct 30 19:16:06.194396 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:06.194264 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" podUID="b0479234-1db6-465f-9454-54d777b09538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" Oct 30 19:16:06.687483 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:06.687448 5894 generic.go:356] "Generic (PLEG): container finished" podID="9d683e67-87f6-4d4b-a208-0ebea816c5a7" containerID="fcc84f87901701cca3cff82cb7b27fa1229e305713851f9d1e94bd8346985db3" exitCode=0 Oct 30 19:16:06.687686 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:06.687494 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp" Oct 30 19:16:06.687686 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:06.687525 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp" event={"ID":"9d683e67-87f6-4d4b-a208-0ebea816c5a7","Type":"ContainerDied","Data":"fcc84f87901701cca3cff82cb7b27fa1229e305713851f9d1e94bd8346985db3"} Oct 30 19:16:06.687686 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:06.687598 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp" event={"ID":"9d683e67-87f6-4d4b-a208-0ebea816c5a7","Type":"ContainerDied","Data":"1bf8ca764547ae5e900f820dd5690b544ae62675536b6716145eafefba27a382"} Oct 30 19:16:06.687686 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:06.687621 5894 scope.go:122] "RemoveContainer" containerID="fcc84f87901701cca3cff82cb7b27fa1229e305713851f9d1e94bd8346985db3" Oct 30 19:16:06.702364 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:06.702339 5894 scope.go:122] "RemoveContainer" containerID="fcc84f87901701cca3cff82cb7b27fa1229e305713851f9d1e94bd8346985db3" Oct 30 19:16:06.703022 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:16:06.702999 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcc84f87901701cca3cff82cb7b27fa1229e305713851f9d1e94bd8346985db3\": container with ID starting with fcc84f87901701cca3cff82cb7b27fa1229e305713851f9d1e94bd8346985db3 not found: ID does not exist" containerID="fcc84f87901701cca3cff82cb7b27fa1229e305713851f9d1e94bd8346985db3" Oct 30 19:16:06.703113 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:06.703033 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcc84f87901701cca3cff82cb7b27fa1229e305713851f9d1e94bd8346985db3"} err="failed to get container status \"fcc84f87901701cca3cff82cb7b27fa1229e305713851f9d1e94bd8346985db3\": rpc error: code = NotFound desc = could not find container \"fcc84f87901701cca3cff82cb7b27fa1229e305713851f9d1e94bd8346985db3\": container with ID starting with fcc84f87901701cca3cff82cb7b27fa1229e305713851f9d1e94bd8346985db3 not found: ID does not exist" Oct 30 19:16:06.872250 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:06.872213 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp"] Oct 30 19:16:06.875462 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:06.875428 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-8da6e2-6d7d48c655-k9kqp"] Oct 30 19:16:07.194021 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:07.193987 5894 patch_prober.go:29] interesting pod/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body= Oct 30 19:16:07.194430 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:07.194049 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" podUID="b0479234-1db6-465f-9454-54d777b09538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" Oct 30 19:16:07.659071 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:07.659036 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9d683e67-87f6-4d4b-a208-0ebea816c5a7" path="/var/lib/kubelet/pods/9d683e67-87f6-4d4b-a208-0ebea816c5a7/volumes" Oct 30 19:16:08.193504 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:08.193471 5894 patch_prober.go:29] interesting pod/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body= Oct 30 19:16:08.193707 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:08.193529 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" podUID="b0479234-1db6-465f-9454-54d777b09538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" Oct 30 19:16:09.198981 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:09.198944 5894 patch_prober.go:29] interesting pod/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" start-of-body= Oct 30 19:16:09.199957 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:09.199928 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" podUID="b0479234-1db6-465f-9454-54d777b09538" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.113:15021/healthz/ready\": dial tcp 10.132.0.113:15021: connect: connection refused" Oct 30 19:16:09.239248 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:09.239154 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:10.219903 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:10.219866 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:16:10.221990 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:10.221964 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:16:10.235352 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:10.235322 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:16:11.040741 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:11.040705 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:11.350653 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:11.350561 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:11.537360 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:11.537326 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:11.988836 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:11.988802 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:14.277599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:14.255415 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:14.470046 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:14.470010 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:16.502241 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:16.501211 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:18.038023 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:18.037987 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:18.253793 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:18.253760 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:18.303219 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:18.303132 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg"] Oct 30 19:16:18.303397 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:18.303330 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" podUID="8a237b2f-38a2-47b6-a0a4-3d89b557b419" containerName="istio-proxy" containerID="cri-o://54e1e2ed45877249f1fa42aee46f3072689369d78d35aa7be2a00a0b2dd1bf0d" gracePeriod=30 Oct 30 19:16:18.449597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:18.446197 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:20.131678 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:16:20.131608 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:16:21.718753 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:21.718716 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:23.573427 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.573360 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:16:23.688066 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.688038 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istiod-ca-cert\") pod \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " Oct 30 19:16:23.688238 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.688108 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-token\" (UniqueName: \"kubernetes.io/projected/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-token\") pod \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " Oct 30 19:16:23.688238 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.688209 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-credential-socket\") pod \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " Oct 30 19:16:23.688363 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.688246 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-workload-certs\") pod \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " Oct 30 19:16:23.688363 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.688278 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8a237b2f-38a2-47b6-a0a4-3d89b557b419-kube-api-access-r7788\" (UniqueName: \"kubernetes.io/projected/8a237b2f-38a2-47b6-a0a4-3d89b557b419-kube-api-access-r7788\") pod \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " Oct 30 19:16:23.688363 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.688339 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-envoy\") pod \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " Oct 30 19:16:23.688517 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.688380 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-podinfo\") pod \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " Oct 30 19:16:23.688517 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.688435 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-data\") pod \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " Oct 30 19:16:23.688678 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.688531 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-workload-socket\") pod \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\" (UID: \"8a237b2f-38a2-47b6-a0a4-3d89b557b419\") " Oct 30 19:16:23.688678 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.688556 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-credential-socket" pod "8a237b2f-38a2-47b6-a0a4-3d89b557b419" (UID: "8a237b2f-38a2-47b6-a0a4-3d89b557b419"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:16:23.688678 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.688597 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istiod-ca-cert" pod "8a237b2f-38a2-47b6-a0a4-3d89b557b419" (UID: "8a237b2f-38a2-47b6-a0a4-3d89b557b419"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:16:23.688678 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.688606 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-workload-certs" pod "8a237b2f-38a2-47b6-a0a4-3d89b557b419" (UID: "8a237b2f-38a2-47b6-a0a4-3d89b557b419"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:16:23.688895 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.688788 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-data" pod "8a237b2f-38a2-47b6-a0a4-3d89b557b419" (UID: "8a237b2f-38a2-47b6-a0a4-3d89b557b419"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:16:23.688954 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.688929 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-workload-socket" pod "8a237b2f-38a2-47b6-a0a4-3d89b557b419" (UID: "8a237b2f-38a2-47b6-a0a4-3d89b557b419"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:16:23.689139 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.689113 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:23.689272 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.689143 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:23.689272 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.689161 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:23.689272 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.689176 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:23.689272 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.689190 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:23.690694 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.690666 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-token" pod "8a237b2f-38a2-47b6-a0a4-3d89b557b419" (UID: "8a237b2f-38a2-47b6-a0a4-3d89b557b419"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:16:23.690787 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.690728 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a237b2f-38a2-47b6-a0a4-3d89b557b419-kube-api-access-r7788" pod "8a237b2f-38a2-47b6-a0a4-3d89b557b419" (UID: "8a237b2f-38a2-47b6-a0a4-3d89b557b419"). InnerVolumeSpecName "kube-api-access-r7788". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:16:23.691003 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.690976 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-podinfo" pod "8a237b2f-38a2-47b6-a0a4-3d89b557b419" (UID: "8a237b2f-38a2-47b6-a0a4-3d89b557b419"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:16:23.691092 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.690977 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-envoy" pod "8a237b2f-38a2-47b6-a0a4-3d89b557b419" (UID: "8a237b2f-38a2-47b6-a0a4-3d89b557b419"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:16:23.790629 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.790561 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:23.790629 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.790625 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r7788\" (UniqueName: \"kubernetes.io/projected/8a237b2f-38a2-47b6-a0a4-3d89b557b419-kube-api-access-r7788\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:23.790839 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.790639 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:23.790839 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.790648 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8a237b2f-38a2-47b6-a0a4-3d89b557b419-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:16:23.915997 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.915963 5894 generic.go:356] "Generic (PLEG): container finished" podID="8a237b2f-38a2-47b6-a0a4-3d89b557b419" containerID="54e1e2ed45877249f1fa42aee46f3072689369d78d35aa7be2a00a0b2dd1bf0d" exitCode=0 Oct 30 19:16:23.916194 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.916016 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" Oct 30 19:16:23.916194 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.916041 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" event={"ID":"8a237b2f-38a2-47b6-a0a4-3d89b557b419","Type":"ContainerDied","Data":"54e1e2ed45877249f1fa42aee46f3072689369d78d35aa7be2a00a0b2dd1bf0d"} Oct 30 19:16:23.916194 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.916107 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg" event={"ID":"8a237b2f-38a2-47b6-a0a4-3d89b557b419","Type":"ContainerDied","Data":"4602d1292a3ce03c6607c88796a34876931c74a5b18ba382948b6023333f85b5"} Oct 30 19:16:23.916194 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.916137 5894 scope.go:122] "RemoveContainer" containerID="54e1e2ed45877249f1fa42aee46f3072689369d78d35aa7be2a00a0b2dd1bf0d" Oct 30 19:16:23.927732 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.927711 5894 scope.go:122] "RemoveContainer" containerID="398d5698699ffc986b484bf657c390586ca5833ca8ee51e124ca58cc0ff27cc8" Oct 30 19:16:23.936811 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.936792 5894 scope.go:122] "RemoveContainer" containerID="54e1e2ed45877249f1fa42aee46f3072689369d78d35aa7be2a00a0b2dd1bf0d" Oct 30 19:16:23.937070 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:16:23.937052 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54e1e2ed45877249f1fa42aee46f3072689369d78d35aa7be2a00a0b2dd1bf0d\": container with ID starting with 54e1e2ed45877249f1fa42aee46f3072689369d78d35aa7be2a00a0b2dd1bf0d not found: ID does not exist" containerID="54e1e2ed45877249f1fa42aee46f3072689369d78d35aa7be2a00a0b2dd1bf0d" Oct 30 19:16:23.937118 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.937080 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54e1e2ed45877249f1fa42aee46f3072689369d78d35aa7be2a00a0b2dd1bf0d"} err="failed to get container status \"54e1e2ed45877249f1fa42aee46f3072689369d78d35aa7be2a00a0b2dd1bf0d\": rpc error: code = NotFound desc = could not find container \"54e1e2ed45877249f1fa42aee46f3072689369d78d35aa7be2a00a0b2dd1bf0d\": container with ID starting with 54e1e2ed45877249f1fa42aee46f3072689369d78d35aa7be2a00a0b2dd1bf0d not found: ID does not exist" Oct 30 19:16:23.937118 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.937097 5894 scope.go:122] "RemoveContainer" containerID="398d5698699ffc986b484bf657c390586ca5833ca8ee51e124ca58cc0ff27cc8" Oct 30 19:16:23.937321 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:16:23.937301 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"398d5698699ffc986b484bf657c390586ca5833ca8ee51e124ca58cc0ff27cc8\": container with ID starting with 398d5698699ffc986b484bf657c390586ca5833ca8ee51e124ca58cc0ff27cc8 not found: ID does not exist" containerID="398d5698699ffc986b484bf657c390586ca5833ca8ee51e124ca58cc0ff27cc8" Oct 30 19:16:23.937376 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:23.937330 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"398d5698699ffc986b484bf657c390586ca5833ca8ee51e124ca58cc0ff27cc8"} err="failed to get container status \"398d5698699ffc986b484bf657c390586ca5833ca8ee51e124ca58cc0ff27cc8\": rpc error: code = NotFound desc = could not find container \"398d5698699ffc986b484bf657c390586ca5833ca8ee51e124ca58cc0ff27cc8\": container with ID starting with 398d5698699ffc986b484bf657c390586ca5833ca8ee51e124ca58cc0ff27cc8 not found: ID does not exist" Oct 30 19:16:24.080549 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:24.080516 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg"] Oct 30 19:16:24.087793 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:24.087767 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-4ebeba-openshift-default-78ccf5f9c5-xjzcg"] Oct 30 19:16:24.766883 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:24.766847 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:24.965787 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:24.965754 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:25.653311 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:25.653276 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8a237b2f-38a2-47b6-a0a4-3d89b557b419" path="/var/lib/kubelet/pods/8a237b2f-38a2-47b6-a0a4-3d89b557b419/volumes" Oct 30 19:16:25.763493 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:25.763455 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:26.300282 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:26.300250 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:26.790818 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:26.790778 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:28.629031 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:28.628993 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:28.887534 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:28.887456 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:30.733449 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:30.733415 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:30.918026 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:30.917117 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:31.849131 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:31.849089 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:32.928301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:32.928270 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:33.124904 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:33.124871 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:33.394978 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:33.394947 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:36.736957 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:36.736921 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:36.915929 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:36.915896 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:37.188895 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:37.188861 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:42.111115 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:42.108150 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:46.774370 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:46.774330 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:50.118674 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:16:50.118630 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:16:51.427537 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:51.427500 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:16:56.214270 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:16:56.214212 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:00.152320 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:00.152284 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz"] Oct 30 19:17:00.152771 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:00.152494 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" podUID="787838ab-7e4d-423b-be79-b34816b39224" containerName="maas-api" containerID="cri-o://94da0f6be9dd79a78d31f2cf02d7d7b4dc999988a01125032b948b9aa28a965f" gracePeriod=30 Oct 30 19:17:00.302443 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:00.302403 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:00.553970 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:00.553937 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" Oct 30 19:17:00.573608 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:00.571541 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/787838ab-7e4d-423b-be79-b34816b39224-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/787838ab-7e4d-423b-be79-b34816b39224-maas-api-tls\") pod \"787838ab-7e4d-423b-be79-b34816b39224\" (UID: \"787838ab-7e4d-423b-be79-b34816b39224\") " Oct 30 19:17:00.573608 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:00.571699 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/787838ab-7e4d-423b-be79-b34816b39224-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/787838ab-7e4d-423b-be79-b34816b39224-openshift-service-ca\") pod \"787838ab-7e4d-423b-be79-b34816b39224\" (UID: \"787838ab-7e4d-423b-be79-b34816b39224\") " Oct 30 19:17:00.573608 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:00.571753 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/787838ab-7e4d-423b-be79-b34816b39224-metrics-certs\" (UniqueName: \"kubernetes.io/secret/787838ab-7e4d-423b-be79-b34816b39224-metrics-certs\") pod \"787838ab-7e4d-423b-be79-b34816b39224\" (UID: \"787838ab-7e4d-423b-be79-b34816b39224\") " Oct 30 19:17:00.573608 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:00.571874 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/787838ab-7e4d-423b-be79-b34816b39224-kube-api-access-7xnbb\" (UniqueName: \"kubernetes.io/projected/787838ab-7e4d-423b-be79-b34816b39224-kube-api-access-7xnbb\") pod \"787838ab-7e4d-423b-be79-b34816b39224\" (UID: \"787838ab-7e4d-423b-be79-b34816b39224\") " Oct 30 19:17:00.582196 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:00.582163 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/787838ab-7e4d-423b-be79-b34816b39224-openshift-service-ca" pod "787838ab-7e4d-423b-be79-b34816b39224" (UID: "787838ab-7e4d-423b-be79-b34816b39224"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:17:00.615553 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:00.612954 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/787838ab-7e4d-423b-be79-b34816b39224-kube-api-access-7xnbb" pod "787838ab-7e4d-423b-be79-b34816b39224" (UID: "787838ab-7e4d-423b-be79-b34816b39224"). InnerVolumeSpecName "kube-api-access-7xnbb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:17:00.623087 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:00.622707 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/787838ab-7e4d-423b-be79-b34816b39224-maas-api-tls" pod "787838ab-7e4d-423b-be79-b34816b39224" (UID: "787838ab-7e4d-423b-be79-b34816b39224"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:17:00.626654 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:00.625133 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/787838ab-7e4d-423b-be79-b34816b39224-metrics-certs" pod "787838ab-7e4d-423b-be79-b34816b39224" (UID: "787838ab-7e4d-423b-be79-b34816b39224"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:17:00.675751 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:00.673712 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/787838ab-7e4d-423b-be79-b34816b39224-maas-api-tls\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:00.675751 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:00.673752 5894 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/787838ab-7e4d-423b-be79-b34816b39224-openshift-service-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:00.675751 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:00.673770 5894 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/787838ab-7e4d-423b-be79-b34816b39224-metrics-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:00.675751 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:00.673783 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xnbb\" (UniqueName: \"kubernetes.io/projected/787838ab-7e4d-423b-be79-b34816b39224-kube-api-access-7xnbb\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:01.237047 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:01.237013 5894 generic.go:356] "Generic (PLEG): container finished" podID="787838ab-7e4d-423b-be79-b34816b39224" containerID="94da0f6be9dd79a78d31f2cf02d7d7b4dc999988a01125032b948b9aa28a965f" exitCode=0 Oct 30 19:17:01.237692 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:01.237060 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" Oct 30 19:17:01.237692 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:01.237095 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" event={"ID":"787838ab-7e4d-423b-be79-b34816b39224","Type":"ContainerDied","Data":"94da0f6be9dd79a78d31f2cf02d7d7b4dc999988a01125032b948b9aa28a965f"} Oct 30 19:17:01.237692 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:01.237154 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz" event={"ID":"787838ab-7e4d-423b-be79-b34816b39224","Type":"ContainerDied","Data":"7f38393ae6b4f583d39e16d919c5fde995abe3a903d96faef749a963a8d54628"} Oct 30 19:17:01.237692 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:01.237181 5894 scope.go:122] "RemoveContainer" containerID="94da0f6be9dd79a78d31f2cf02d7d7b4dc999988a01125032b948b9aa28a965f" Oct 30 19:17:01.250383 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:01.250363 5894 scope.go:122] "RemoveContainer" containerID="94da0f6be9dd79a78d31f2cf02d7d7b4dc999988a01125032b948b9aa28a965f" Oct 30 19:17:01.250771 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:17:01.250732 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94da0f6be9dd79a78d31f2cf02d7d7b4dc999988a01125032b948b9aa28a965f\": container with ID starting with 94da0f6be9dd79a78d31f2cf02d7d7b4dc999988a01125032b948b9aa28a965f not found: ID does not exist" containerID="94da0f6be9dd79a78d31f2cf02d7d7b4dc999988a01125032b948b9aa28a965f" Oct 30 19:17:01.250875 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:01.250783 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94da0f6be9dd79a78d31f2cf02d7d7b4dc999988a01125032b948b9aa28a965f"} err="failed to get container status \"94da0f6be9dd79a78d31f2cf02d7d7b4dc999988a01125032b948b9aa28a965f\": rpc error: code = NotFound desc = could not find container \"94da0f6be9dd79a78d31f2cf02d7d7b4dc999988a01125032b948b9aa28a965f\": container with ID starting with 94da0f6be9dd79a78d31f2cf02d7d7b4dc999988a01125032b948b9aa28a965f not found: ID does not exist" Oct 30 19:17:01.405310 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:01.405271 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz"] Oct 30 19:17:01.410056 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:01.410030 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-f5763a-66d659bc6-xrdwz"] Oct 30 19:17:01.471366 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:01.471279 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:01.654603 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:01.653563 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="787838ab-7e4d-423b-be79-b34816b39224" path="/var/lib/kubelet/pods/787838ab-7e4d-423b-be79-b34816b39224/volumes" Oct 30 19:17:01.846541 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:01.846437 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:02.175101 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:02.175065 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:02.354669 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:02.354624 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4"] Oct 30 19:17:02.367170 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:02.367123 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-f5763a-4dwk4"] Oct 30 19:17:02.821744 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:02.821710 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:03.049051 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:03.049019 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:03.290733 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:03.290699 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:03.477014 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:03.476978 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:03.651180 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:03.651145 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d208721-b481-4c47-8ca5-da1f7edab8ac" path="/var/lib/kubelet/pods/7d208721-b481-4c47-8ca5-da1f7edab8ac/volumes" Oct 30 19:17:04.095454 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:04.095374 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:07.646929 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:07.646893 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:08.714828 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:08.714790 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:08.899457 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:08.899428 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:09.039520 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:09.039432 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5"] Oct 30 19:17:09.039735 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:09.039707 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" podUID="c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" containerName="istio-proxy" containerID="cri-o://1c6cbfb9bdc0ecee30c44d556524a839b0d371f5e7200ddae3cb9371b931499c" gracePeriod=30 Oct 30 19:17:09.150228 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:09.150191 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:09.372167 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:09.372086 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:09.571825 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:09.571773 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:10.138329 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:10.138297 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:10.445287 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:10.445256 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:10.634485 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:10.634455 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:10.830630 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:10.830551 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:11.879615 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:11.879557 5894 patch_prober.go:29] interesting pod/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:17:11.880026 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:11.879653 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" podUID="c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:17:12.666907 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:12.666852 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:14.307499 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.307435 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 19:17:14.320747 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.320716 5894 generic.go:356] "Generic (PLEG): container finished" podID="c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" containerID="1c6cbfb9bdc0ecee30c44d556524a839b0d371f5e7200ddae3cb9371b931499c" exitCode=0 Oct 30 19:17:14.320887 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.320755 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" Oct 30 19:17:14.320887 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.320801 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" event={"ID":"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5","Type":"ContainerDied","Data":"1c6cbfb9bdc0ecee30c44d556524a839b0d371f5e7200ddae3cb9371b931499c"} Oct 30 19:17:14.320887 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.320862 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5" event={"ID":"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5","Type":"ContainerDied","Data":"c4b54f84f6cf9424afc0e0d1dfb117c05652bc257aad6991ec917f6eeb22c577"} Oct 30 19:17:14.321061 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.320892 5894 scope.go:122] "RemoveContainer" containerID="1c6cbfb9bdc0ecee30c44d556524a839b0d371f5e7200ddae3cb9371b931499c" Oct 30 19:17:14.330689 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.330672 5894 scope.go:122] "RemoveContainer" containerID="1c6cbfb9bdc0ecee30c44d556524a839b0d371f5e7200ddae3cb9371b931499c" Oct 30 19:17:14.330952 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:17:14.330932 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1c6cbfb9bdc0ecee30c44d556524a839b0d371f5e7200ddae3cb9371b931499c\": container with ID starting with 1c6cbfb9bdc0ecee30c44d556524a839b0d371f5e7200ddae3cb9371b931499c not found: ID does not exist" containerID="1c6cbfb9bdc0ecee30c44d556524a839b0d371f5e7200ddae3cb9371b931499c" Oct 30 19:17:14.330995 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.330960 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1c6cbfb9bdc0ecee30c44d556524a839b0d371f5e7200ddae3cb9371b931499c"} err="failed to get container status \"1c6cbfb9bdc0ecee30c44d556524a839b0d371f5e7200ddae3cb9371b931499c\": rpc error: code = NotFound desc = could not find container \"1c6cbfb9bdc0ecee30c44d556524a839b0d371f5e7200ddae3cb9371b931499c\": container with ID starting with 1c6cbfb9bdc0ecee30c44d556524a839b0d371f5e7200ddae3cb9371b931499c not found: ID does not exist" Oct 30 19:17:14.410955 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.410930 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-envoy\") pod \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " Oct 30 19:17:14.411112 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.411003 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istiod-ca-cert\") pod \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " Oct 30 19:17:14.411189 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.411129 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-podinfo\") pod \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " Oct 30 19:17:14.411189 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.411159 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-data\") pod \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " Oct 30 19:17:14.411298 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.411216 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-token\" (UniqueName: \"kubernetes.io/projected/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-token\") pod \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " Oct 30 19:17:14.411298 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.411269 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-kube-api-access-rbj9m\" (UniqueName: \"kubernetes.io/projected/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-kube-api-access-rbj9m\") pod \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " Oct 30 19:17:14.411298 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.411291 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-workload-certs\") pod \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " Oct 30 19:17:14.411471 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.411324 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-credential-socket\") pod \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " Oct 30 19:17:14.411471 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.411363 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-workload-socket\") pod \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\" (UID: \"c7acd06a-8759-4b0a-9e4e-7874e9fd38e5\") " Oct 30 19:17:14.411471 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.411453 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istiod-ca-cert" pod "c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" (UID: "c7acd06a-8759-4b0a-9e4e-7874e9fd38e5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:17:14.411706 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.411677 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-workload-certs" pod "c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" (UID: "c7acd06a-8759-4b0a-9e4e-7874e9fd38e5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:14.411799 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.411720 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-data" pod "c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" (UID: "c7acd06a-8759-4b0a-9e4e-7874e9fd38e5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:14.411799 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.411749 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-credential-socket" pod "c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" (UID: "c7acd06a-8759-4b0a-9e4e-7874e9fd38e5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:14.411945 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.411926 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-workload-socket" pod "c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" (UID: "c7acd06a-8759-4b0a-9e4e-7874e9fd38e5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:14.412062 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.412043 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:14.412109 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.412065 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:14.412109 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.412074 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:14.412109 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.412083 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:14.412109 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.412091 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:14.413247 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.413224 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-envoy" pod "c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" (UID: "c7acd06a-8759-4b0a-9e4e-7874e9fd38e5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:14.413536 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.413519 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-podinfo" pod "c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" (UID: "c7acd06a-8759-4b0a-9e4e-7874e9fd38e5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:17:14.413880 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.413851 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-kube-api-access-rbj9m" pod "c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" (UID: "c7acd06a-8759-4b0a-9e4e-7874e9fd38e5"). InnerVolumeSpecName "kube-api-access-rbj9m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:17:14.413997 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.413933 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-token" pod "c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" (UID: "c7acd06a-8759-4b0a-9e4e-7874e9fd38e5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:17:14.513809 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.513771 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:14.513809 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.513805 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:14.513809 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.513815 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbj9m\" (UniqueName: \"kubernetes.io/projected/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-kube-api-access-rbj9m\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:14.514104 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.513826 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:14.713107 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.713042 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:14.755382 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.755350 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5"] Oct 30 19:17:14.759352 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:14.759328 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-f5763a-openshift-default-545765b89-tbvj5"] Oct 30 19:17:15.651956 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:15.651923 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" path="/var/lib/kubelet/pods/c7acd06a-8759-4b0a-9e4e-7874e9fd38e5/volumes" Oct 30 19:17:16.593881 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:16.593848 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:16.769350 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:16.769305 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:16.963271 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:16.961080 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:17.626893 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:17.626852 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l"] Oct 30 19:17:17.765361 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:17.765332 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:18.889994 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:18.889961 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv"] Oct 30 19:17:18.890403 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:18.890164 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" podUID="03ef4756-9a48-4f73-9229-6e7cc936e052" containerName="maas-api" containerID="cri-o://7727b606e714ba69ec156d47f2dff87a0d9d2f4107f75ab15b595a4568597bf8" gracePeriod=30 Oct 30 19:17:19.031850 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.031811 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:19.232162 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.232135 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" Oct 30 19:17:19.286421 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.283992 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/03ef4756-9a48-4f73-9229-6e7cc936e052-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03ef4756-9a48-4f73-9229-6e7cc936e052-openshift-service-ca\") pod \"03ef4756-9a48-4f73-9229-6e7cc936e052\" (UID: \"03ef4756-9a48-4f73-9229-6e7cc936e052\") " Oct 30 19:17:19.286421 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.284160 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/03ef4756-9a48-4f73-9229-6e7cc936e052-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03ef4756-9a48-4f73-9229-6e7cc936e052-maas-api-tls\") pod \"03ef4756-9a48-4f73-9229-6e7cc936e052\" (UID: \"03ef4756-9a48-4f73-9229-6e7cc936e052\") " Oct 30 19:17:19.286421 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.284511 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/03ef4756-9a48-4f73-9229-6e7cc936e052-metrics-certs\" (UniqueName: \"kubernetes.io/secret/03ef4756-9a48-4f73-9229-6e7cc936e052-metrics-certs\") pod \"03ef4756-9a48-4f73-9229-6e7cc936e052\" (UID: \"03ef4756-9a48-4f73-9229-6e7cc936e052\") " Oct 30 19:17:19.286421 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.284632 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/03ef4756-9a48-4f73-9229-6e7cc936e052-kube-api-access-bq59v\" (UniqueName: \"kubernetes.io/projected/03ef4756-9a48-4f73-9229-6e7cc936e052-kube-api-access-bq59v\") pod \"03ef4756-9a48-4f73-9229-6e7cc936e052\" (UID: \"03ef4756-9a48-4f73-9229-6e7cc936e052\") " Oct 30 19:17:19.301390 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.301335 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03ef4756-9a48-4f73-9229-6e7cc936e052-openshift-service-ca" pod "03ef4756-9a48-4f73-9229-6e7cc936e052" (UID: "03ef4756-9a48-4f73-9229-6e7cc936e052"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:17:19.358984 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.348029 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03ef4756-9a48-4f73-9229-6e7cc936e052-metrics-certs" pod "03ef4756-9a48-4f73-9229-6e7cc936e052" (UID: "03ef4756-9a48-4f73-9229-6e7cc936e052"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:17:19.358984 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.348889 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03ef4756-9a48-4f73-9229-6e7cc936e052-kube-api-access-bq59v" pod "03ef4756-9a48-4f73-9229-6e7cc936e052" (UID: "03ef4756-9a48-4f73-9229-6e7cc936e052"). InnerVolumeSpecName "kube-api-access-bq59v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:17:19.358984 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.348994 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03ef4756-9a48-4f73-9229-6e7cc936e052-maas-api-tls" pod "03ef4756-9a48-4f73-9229-6e7cc936e052" (UID: "03ef4756-9a48-4f73-9229-6e7cc936e052"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:17:19.395630 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.386646 5894 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/03ef4756-9a48-4f73-9229-6e7cc936e052-metrics-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:19.395630 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.386696 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bq59v\" (UniqueName: \"kubernetes.io/projected/03ef4756-9a48-4f73-9229-6e7cc936e052-kube-api-access-bq59v\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:19.395630 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.386711 5894 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/03ef4756-9a48-4f73-9229-6e7cc936e052-openshift-service-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:19.395630 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.386722 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/03ef4756-9a48-4f73-9229-6e7cc936e052-maas-api-tls\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:19.395630 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.391125 5894 generic.go:356] "Generic (PLEG): container finished" podID="03ef4756-9a48-4f73-9229-6e7cc936e052" containerID="7727b606e714ba69ec156d47f2dff87a0d9d2f4107f75ab15b595a4568597bf8" exitCode=0 Oct 30 19:17:19.395630 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.391372 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" event={"ID":"03ef4756-9a48-4f73-9229-6e7cc936e052","Type":"ContainerDied","Data":"7727b606e714ba69ec156d47f2dff87a0d9d2f4107f75ab15b595a4568597bf8"} Oct 30 19:17:19.395630 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.391428 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" event={"ID":"03ef4756-9a48-4f73-9229-6e7cc936e052","Type":"ContainerDied","Data":"2aecc585c96c963bf3f38124529136913f550a8ec36b11b5257303af9cbb1501"} Oct 30 19:17:19.395630 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.391460 5894 scope.go:122] "RemoveContainer" containerID="7727b606e714ba69ec156d47f2dff87a0d9d2f4107f75ab15b595a4568597bf8" Oct 30 19:17:19.395630 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.391613 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv" Oct 30 19:17:19.428253 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.427930 5894 scope.go:122] "RemoveContainer" containerID="7727b606e714ba69ec156d47f2dff87a0d9d2f4107f75ab15b595a4568597bf8" Oct 30 19:17:19.429022 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:17:19.428495 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7727b606e714ba69ec156d47f2dff87a0d9d2f4107f75ab15b595a4568597bf8\": container with ID starting with 7727b606e714ba69ec156d47f2dff87a0d9d2f4107f75ab15b595a4568597bf8 not found: ID does not exist" containerID="7727b606e714ba69ec156d47f2dff87a0d9d2f4107f75ab15b595a4568597bf8" Oct 30 19:17:19.429022 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.428533 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7727b606e714ba69ec156d47f2dff87a0d9d2f4107f75ab15b595a4568597bf8"} err="failed to get container status \"7727b606e714ba69ec156d47f2dff87a0d9d2f4107f75ab15b595a4568597bf8\": rpc error: code = NotFound desc = could not find container \"7727b606e714ba69ec156d47f2dff87a0d9d2f4107f75ab15b595a4568597bf8\": container with ID starting with 7727b606e714ba69ec156d47f2dff87a0d9d2f4107f75ab15b595a4568597bf8 not found: ID does not exist" Oct 30 19:17:19.564682 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.564565 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv"] Oct 30 19:17:19.569727 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.569691 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-ae6a138d-d4cf57f4b-mqhkv"] Oct 30 19:17:19.650569 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:19.650523 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="03ef4756-9a48-4f73-9229-6e7cc936e052" path="/var/lib/kubelet/pods/03ef4756-9a48-4f73-9229-6e7cc936e052/volumes" Oct 30 19:17:20.119662 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:17:20.119624 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:17:20.486391 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:20.486357 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:20.696912 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:20.696864 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:20.995860 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:20.995821 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:21.633189 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:21.633129 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:22.110784 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:22.110740 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:22.441663 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:22.441633 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:22.628128 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:22.628078 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:23.085160 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:23.085128 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:24.745858 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:24.745793 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:24.933127 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:24.933097 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:25.107699 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:25.107628 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:26.539751 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:26.539717 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s"] Oct 30 19:17:26.654750 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:26.654718 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:27.719596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:27.718140 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:29.159600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:29.159545 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:29.338695 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:29.338661 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:29.450846 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:29.450765 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2"] Oct 30 19:17:29.451024 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:29.450991 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" podUID="2a4529e2-c317-4d6d-916f-b77baf48009a" containerName="istio-proxy" containerID="cri-o://6f4b2b64a18baada6b2d8bad477a01dc946cab7fdbb9b800c3981c9746823b87" gracePeriod=30 Oct 30 19:17:29.600599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:29.599854 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:29.974147 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:29.974111 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:32.049260 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:32.049219 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:32.650157 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:32.650122 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" containerID="cri-o://ac4f072bc9807e52550b3c997115da13b99601b954c6c8b6484cffe0c396b189" gracePeriod=45 Oct 30 19:17:32.934061 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:32.934036 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 19:17:33.046600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.040286 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-home\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-home\") pod \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " Oct 30 19:17:33.046600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.040364 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-dshm\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-dshm\") pod \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " Oct 30 19:17:33.046600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.040474 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-tmp-dir\") pod \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " Oct 30 19:17:33.046600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.040560 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-model-cache\") pod \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " Oct 30 19:17:33.046600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.040611 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-tls-certs\" (UniqueName: \"kubernetes.io/secret/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-tls-certs\") pod \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " Oct 30 19:17:33.046600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.040712 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-kube-api-access-jjhxm\" (UniqueName: \"kubernetes.io/projected/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-kube-api-access-jjhxm\") pod \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\" (UID: \"67c4ee9e-c041-4f48-936b-afb5a69dbd1f\") " Oct 30 19:17:33.046600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.041931 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-home" pod "67c4ee9e-c041-4f48-936b-afb5a69dbd1f" (UID: "67c4ee9e-c041-4f48-936b-afb5a69dbd1f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:33.046600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.045198 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-tmp-dir" pod "67c4ee9e-c041-4f48-936b-afb5a69dbd1f" (UID: "67c4ee9e-c041-4f48-936b-afb5a69dbd1f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:33.046600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.045402 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-model-cache" pod "67c4ee9e-c041-4f48-936b-afb5a69dbd1f" (UID: "67c4ee9e-c041-4f48-936b-afb5a69dbd1f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:33.046600 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.046236 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-kube-api-access-jjhxm" pod "67c4ee9e-c041-4f48-936b-afb5a69dbd1f" (UID: "67c4ee9e-c041-4f48-936b-afb5a69dbd1f"). InnerVolumeSpecName "kube-api-access-jjhxm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:17:33.050783 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.047896 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-tls-certs" pod "67c4ee9e-c041-4f48-936b-afb5a69dbd1f" (UID: "67c4ee9e-c041-4f48-936b-afb5a69dbd1f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:17:33.050783 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.049681 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-dshm" pod "67c4ee9e-c041-4f48-936b-afb5a69dbd1f" (UID: "67c4ee9e-c041-4f48-936b-afb5a69dbd1f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:33.143029 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.142991 5894 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-home\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:33.143029 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.143028 5894 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-dshm\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:33.143029 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.143038 5894 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-tmp-dir\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:33.143348 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.143047 5894 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-model-cache\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:33.143348 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.143056 5894 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-tls-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:33.143348 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.143067 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jjhxm\" (UniqueName: \"kubernetes.io/projected/67c4ee9e-c041-4f48-936b-afb5a69dbd1f-kube-api-access-jjhxm\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:33.515939 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.515904 5894 generic.go:356] "Generic (PLEG): container finished" podID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerID="ac4f072bc9807e52550b3c997115da13b99601b954c6c8b6484cffe0c396b189" exitCode=0 Oct 30 19:17:33.516097 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.515943 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" Oct 30 19:17:33.516097 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.515951 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" event={"ID":"67c4ee9e-c041-4f48-936b-afb5a69dbd1f","Type":"ContainerDied","Data":"ac4f072bc9807e52550b3c997115da13b99601b954c6c8b6484cffe0c396b189"} Oct 30 19:17:33.516097 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.515982 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l" event={"ID":"67c4ee9e-c041-4f48-936b-afb5a69dbd1f","Type":"ContainerDied","Data":"d32598ac3378af5683bb4b65e4a4b21991c8f8919361e4fb9c7ed7422fdab91b"} Oct 30 19:17:33.516097 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.515999 5894 scope.go:122] "RemoveContainer" containerID="ac4f072bc9807e52550b3c997115da13b99601b954c6c8b6484cffe0c396b189" Oct 30 19:17:33.527756 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.527735 5894 scope.go:122] "RemoveContainer" containerID="ac4f072bc9807e52550b3c997115da13b99601b954c6c8b6484cffe0c396b189" Oct 30 19:17:33.528063 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:17:33.528043 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac4f072bc9807e52550b3c997115da13b99601b954c6c8b6484cffe0c396b189\": container with ID starting with ac4f072bc9807e52550b3c997115da13b99601b954c6c8b6484cffe0c396b189 not found: ID does not exist" containerID="ac4f072bc9807e52550b3c997115da13b99601b954c6c8b6484cffe0c396b189" Oct 30 19:17:33.528107 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.528074 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac4f072bc9807e52550b3c997115da13b99601b954c6c8b6484cffe0c396b189"} err="failed to get container status \"ac4f072bc9807e52550b3c997115da13b99601b954c6c8b6484cffe0c396b189\": rpc error: code = NotFound desc = could not find container \"ac4f072bc9807e52550b3c997115da13b99601b954c6c8b6484cffe0c396b189\": container with ID starting with ac4f072bc9807e52550b3c997115da13b99601b954c6c8b6484cffe0c396b189 not found: ID does not exist" Oct 30 19:17:33.684980 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.684939 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l"] Oct 30 19:17:33.721614 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.721552 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:33.732024 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.731998 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w6-41db7c/distinct-w6-41db7c-kserve-5fc49465d4-b9g7l"] Oct 30 19:17:33.914613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:33.914541 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:34.142473 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.142418 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:34.332005 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.331928 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:34.726774 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.726749 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:17:34.744074 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.744044 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:34.865145 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.865118 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-token\" (UniqueName: \"kubernetes.io/projected/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-token\") pod \"2a4529e2-c317-4d6d-916f-b77baf48009a\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " Oct 30 19:17:34.865331 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.865169 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-envoy\") pod \"2a4529e2-c317-4d6d-916f-b77baf48009a\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " Oct 30 19:17:34.865331 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.865187 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/2a4529e2-c317-4d6d-916f-b77baf48009a-kube-api-access-dx9zb\" (UniqueName: \"kubernetes.io/projected/2a4529e2-c317-4d6d-916f-b77baf48009a-kube-api-access-dx9zb\") pod \"2a4529e2-c317-4d6d-916f-b77baf48009a\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " Oct 30 19:17:34.865331 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.865219 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/2a4529e2-c317-4d6d-916f-b77baf48009a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a4529e2-c317-4d6d-916f-b77baf48009a-istiod-ca-cert\") pod \"2a4529e2-c317-4d6d-916f-b77baf48009a\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " Oct 30 19:17:34.865331 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.865270 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-credential-socket\") pod \"2a4529e2-c317-4d6d-916f-b77baf48009a\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " Oct 30 19:17:34.865331 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.865296 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-workload-socket\") pod \"2a4529e2-c317-4d6d-916f-b77baf48009a\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " Oct 30 19:17:34.865644 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.865351 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-data\") pod \"2a4529e2-c317-4d6d-916f-b77baf48009a\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " Oct 30 19:17:34.865644 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.865517 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-workload-certs\") pod \"2a4529e2-c317-4d6d-916f-b77baf48009a\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " Oct 30 19:17:34.865644 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.865629 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-podinfo\") pod \"2a4529e2-c317-4d6d-916f-b77baf48009a\" (UID: \"2a4529e2-c317-4d6d-916f-b77baf48009a\") " Oct 30 19:17:34.865824 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.865629 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-credential-socket" pod "2a4529e2-c317-4d6d-916f-b77baf48009a" (UID: "2a4529e2-c317-4d6d-916f-b77baf48009a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:34.865824 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.865720 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-workload-socket" pod "2a4529e2-c317-4d6d-916f-b77baf48009a" (UID: "2a4529e2-c317-4d6d-916f-b77baf48009a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:34.865824 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.865782 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a4529e2-c317-4d6d-916f-b77baf48009a-istiod-ca-cert" pod "2a4529e2-c317-4d6d-916f-b77baf48009a" (UID: "2a4529e2-c317-4d6d-916f-b77baf48009a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:17:34.865980 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.865832 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-workload-certs" pod "2a4529e2-c317-4d6d-916f-b77baf48009a" (UID: "2a4529e2-c317-4d6d-916f-b77baf48009a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:34.866029 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.865996 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-data" pod "2a4529e2-c317-4d6d-916f-b77baf48009a" (UID: "2a4529e2-c317-4d6d-916f-b77baf48009a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:34.866223 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.866203 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:34.866303 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.866225 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a4529e2-c317-4d6d-916f-b77baf48009a-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:34.866303 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.866239 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:34.866303 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.866266 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:34.866303 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.866281 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:34.867643 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.867619 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-token" pod "2a4529e2-c317-4d6d-916f-b77baf48009a" (UID: "2a4529e2-c317-4d6d-916f-b77baf48009a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:17:34.867875 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.867851 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-podinfo" pod "2a4529e2-c317-4d6d-916f-b77baf48009a" (UID: "2a4529e2-c317-4d6d-916f-b77baf48009a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:17:34.867967 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.867946 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-envoy" pod "2a4529e2-c317-4d6d-916f-b77baf48009a" (UID: "2a4529e2-c317-4d6d-916f-b77baf48009a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:34.868033 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.867978 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a4529e2-c317-4d6d-916f-b77baf48009a-kube-api-access-dx9zb" pod "2a4529e2-c317-4d6d-916f-b77baf48009a" (UID: "2a4529e2-c317-4d6d-916f-b77baf48009a"). InnerVolumeSpecName "kube-api-access-dx9zb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:17:34.896014 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.895991 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g"] Oct 30 19:17:34.968029 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.967995 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:34.968221 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.968140 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:34.968221 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.968159 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a4529e2-c317-4d6d-916f-b77baf48009a-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:34.968221 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:34.968175 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dx9zb\" (UniqueName: \"kubernetes.io/projected/2a4529e2-c317-4d6d-916f-b77baf48009a-kube-api-access-dx9zb\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:35.059458 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:35.059399 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:35.530731 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:35.530697 5894 generic.go:356] "Generic (PLEG): container finished" podID="2a4529e2-c317-4d6d-916f-b77baf48009a" containerID="6f4b2b64a18baada6b2d8bad477a01dc946cab7fdbb9b800c3981c9746823b87" exitCode=0 Oct 30 19:17:35.531221 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:35.530741 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" Oct 30 19:17:35.531221 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:35.530769 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" event={"ID":"2a4529e2-c317-4d6d-916f-b77baf48009a","Type":"ContainerDied","Data":"6f4b2b64a18baada6b2d8bad477a01dc946cab7fdbb9b800c3981c9746823b87"} Oct 30 19:17:35.531221 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:35.530838 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2" event={"ID":"2a4529e2-c317-4d6d-916f-b77baf48009a","Type":"ContainerDied","Data":"054c76e07ad03af49b1365967b1a930f225242319ffe80bc6cd5483d0ff89d48"} Oct 30 19:17:35.531221 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:35.530870 5894 scope.go:122] "RemoveContainer" containerID="6f4b2b64a18baada6b2d8bad477a01dc946cab7fdbb9b800c3981c9746823b87" Oct 30 19:17:35.542310 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:35.542294 5894 scope.go:122] "RemoveContainer" containerID="6f68a8b0b7791d38ab0ecdac757da4175e5501417c6265b6ef12dd5c03881a00" Oct 30 19:17:35.551358 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:35.551338 5894 scope.go:122] "RemoveContainer" containerID="6f4b2b64a18baada6b2d8bad477a01dc946cab7fdbb9b800c3981c9746823b87" Oct 30 19:17:35.551634 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:17:35.551614 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f4b2b64a18baada6b2d8bad477a01dc946cab7fdbb9b800c3981c9746823b87\": container with ID starting with 6f4b2b64a18baada6b2d8bad477a01dc946cab7fdbb9b800c3981c9746823b87 not found: ID does not exist" containerID="6f4b2b64a18baada6b2d8bad477a01dc946cab7fdbb9b800c3981c9746823b87" Oct 30 19:17:35.551686 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:35.551644 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f4b2b64a18baada6b2d8bad477a01dc946cab7fdbb9b800c3981c9746823b87"} err="failed to get container status \"6f4b2b64a18baada6b2d8bad477a01dc946cab7fdbb9b800c3981c9746823b87\": rpc error: code = NotFound desc = could not find container \"6f4b2b64a18baada6b2d8bad477a01dc946cab7fdbb9b800c3981c9746823b87\": container with ID starting with 6f4b2b64a18baada6b2d8bad477a01dc946cab7fdbb9b800c3981c9746823b87 not found: ID does not exist" Oct 30 19:17:35.551686 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:35.551661 5894 scope.go:122] "RemoveContainer" containerID="6f68a8b0b7791d38ab0ecdac757da4175e5501417c6265b6ef12dd5c03881a00" Oct 30 19:17:35.551924 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:17:35.551906 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f68a8b0b7791d38ab0ecdac757da4175e5501417c6265b6ef12dd5c03881a00\": container with ID starting with 6f68a8b0b7791d38ab0ecdac757da4175e5501417c6265b6ef12dd5c03881a00 not found: ID does not exist" containerID="6f68a8b0b7791d38ab0ecdac757da4175e5501417c6265b6ef12dd5c03881a00" Oct 30 19:17:35.551978 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:35.551929 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f68a8b0b7791d38ab0ecdac757da4175e5501417c6265b6ef12dd5c03881a00"} err="failed to get container status \"6f68a8b0b7791d38ab0ecdac757da4175e5501417c6265b6ef12dd5c03881a00\": rpc error: code = NotFound desc = could not find container \"6f68a8b0b7791d38ab0ecdac757da4175e5501417c6265b6ef12dd5c03881a00\": container with ID starting with 6f68a8b0b7791d38ab0ecdac757da4175e5501417c6265b6ef12dd5c03881a00 not found: ID does not exist" Oct 30 19:17:35.658737 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:35.658701 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" path="/var/lib/kubelet/pods/67c4ee9e-c041-4f48-936b-afb5a69dbd1f/volumes" Oct 30 19:17:35.703739 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:35.703706 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2"] Oct 30 19:17:35.753313 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:35.753282 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-87ed12-openshift-default-7ddcd7cf97-286g2"] Oct 30 19:17:37.651387 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:37.651355 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="2a4529e2-c317-4d6d-916f-b77baf48009a" path="/var/lib/kubelet/pods/2a4529e2-c317-4d6d-916f-b77baf48009a/volumes" Oct 30 19:17:39.333804 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:39.333764 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:40.724124 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:40.724090 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:40.952459 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:40.952338 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:41.169594 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.165528 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:41.392597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.379899 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:41.555351 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.555277 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:41.560719 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.560685 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" podUID="48092d6e-157f-4df3-828b-497746965c69" containerName="main" containerID="cri-o://f2a9008231c412a37ddd6c059aee1b45bcc4c928cc41aac91d4b94b103844b33" gracePeriod=45 Oct 30 19:17:41.763442 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.763394 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:41.825444 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.825382 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:17:41.875916 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.875886 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/48092d6e-157f-4df3-828b-497746965c69-kube-api-access-cvwrk\" (UniqueName: \"kubernetes.io/projected/48092d6e-157f-4df3-828b-497746965c69-kube-api-access-cvwrk\") pod \"48092d6e-157f-4df3-828b-497746965c69\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " Oct 30 19:17:41.876058 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.875955 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/48092d6e-157f-4df3-828b-497746965c69-tls-certs\" (UniqueName: \"kubernetes.io/secret/48092d6e-157f-4df3-828b-497746965c69-tls-certs\") pod \"48092d6e-157f-4df3-828b-497746965c69\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " Oct 30 19:17:41.876058 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.876033 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-home\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-home\") pod \"48092d6e-157f-4df3-828b-497746965c69\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " Oct 30 19:17:41.876134 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.876062 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-model-cache\") pod \"48092d6e-157f-4df3-828b-497746965c69\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " Oct 30 19:17:41.876134 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.876100 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-dshm\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-dshm\") pod \"48092d6e-157f-4df3-828b-497746965c69\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " Oct 30 19:17:41.876266 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.876141 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-tmp-dir\") pod \"48092d6e-157f-4df3-828b-497746965c69\" (UID: \"48092d6e-157f-4df3-828b-497746965c69\") " Oct 30 19:17:41.876372 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.876341 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-home" pod "48092d6e-157f-4df3-828b-497746965c69" (UID: "48092d6e-157f-4df3-828b-497746965c69"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:41.876372 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.876350 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-model-cache" pod "48092d6e-157f-4df3-828b-497746965c69" (UID: "48092d6e-157f-4df3-828b-497746965c69"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:41.876560 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.876534 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-tmp-dir" pod "48092d6e-157f-4df3-828b-497746965c69" (UID: "48092d6e-157f-4df3-828b-497746965c69"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:41.876770 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.876740 5894 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-home\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:41.876770 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.876768 5894 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-model-cache\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:41.876942 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.876785 5894 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-tmp-dir\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:41.878350 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.878321 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-dshm" pod "48092d6e-157f-4df3-828b-497746965c69" (UID: "48092d6e-157f-4df3-828b-497746965c69"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:41.878444 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.878325 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/48092d6e-157f-4df3-828b-497746965c69-tls-certs" pod "48092d6e-157f-4df3-828b-497746965c69" (UID: "48092d6e-157f-4df3-828b-497746965c69"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:17:41.878444 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.878378 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/48092d6e-157f-4df3-828b-497746965c69-kube-api-access-cvwrk" pod "48092d6e-157f-4df3-828b-497746965c69" (UID: "48092d6e-157f-4df3-828b-497746965c69"). InnerVolumeSpecName "kube-api-access-cvwrk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:17:41.978674 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.978633 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvwrk\" (UniqueName: \"kubernetes.io/projected/48092d6e-157f-4df3-828b-497746965c69-kube-api-access-cvwrk\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:41.978674 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.978673 5894 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/48092d6e-157f-4df3-828b-497746965c69-tls-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:41.978869 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:41.978690 5894 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/48092d6e-157f-4df3-828b-497746965c69-dshm\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:42.587148 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:42.574908 5894 generic.go:356] "Generic (PLEG): container finished" podID="48092d6e-157f-4df3-828b-497746965c69" containerID="f2a9008231c412a37ddd6c059aee1b45bcc4c928cc41aac91d4b94b103844b33" exitCode=0 Oct 30 19:17:42.587148 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:42.575102 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" event={"ID":"48092d6e-157f-4df3-828b-497746965c69","Type":"ContainerDied","Data":"f2a9008231c412a37ddd6c059aee1b45bcc4c928cc41aac91d4b94b103844b33"} Oct 30 19:17:42.587148 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:42.575139 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" event={"ID":"48092d6e-157f-4df3-828b-497746965c69","Type":"ContainerDied","Data":"4c2411c26ad8bcc9d0c8af2f555f401cc9d586f3d4d484c63f58a4be5968bd31"} Oct 30 19:17:42.587148 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:42.575160 5894 scope.go:122] "RemoveContainer" containerID="f2a9008231c412a37ddd6c059aee1b45bcc4c928cc41aac91d4b94b103844b33" Oct 30 19:17:42.587148 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:42.575321 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s" Oct 30 19:17:42.632251 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:42.632231 5894 scope.go:122] "RemoveContainer" containerID="f2a9008231c412a37ddd6c059aee1b45bcc4c928cc41aac91d4b94b103844b33" Oct 30 19:17:42.632540 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:17:42.632520 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2a9008231c412a37ddd6c059aee1b45bcc4c928cc41aac91d4b94b103844b33\": container with ID starting with f2a9008231c412a37ddd6c059aee1b45bcc4c928cc41aac91d4b94b103844b33 not found: ID does not exist" containerID="f2a9008231c412a37ddd6c059aee1b45bcc4c928cc41aac91d4b94b103844b33" Oct 30 19:17:42.632625 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:42.632551 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2a9008231c412a37ddd6c059aee1b45bcc4c928cc41aac91d4b94b103844b33"} err="failed to get container status \"f2a9008231c412a37ddd6c059aee1b45bcc4c928cc41aac91d4b94b103844b33\": rpc error: code = NotFound desc = could not find container \"f2a9008231c412a37ddd6c059aee1b45bcc4c928cc41aac91d4b94b103844b33\": container with ID starting with f2a9008231c412a37ddd6c059aee1b45bcc4c928cc41aac91d4b94b103844b33 not found: ID does not exist" Oct 30 19:17:42.755606 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:42.755259 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s"] Oct 30 19:17:42.797341 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:42.797307 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-ae6a138d/e2e-lifecycle-model-ae6a138d-kserve-5bf467dcbf-kmj9s"] Oct 30 19:17:43.650322 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:43.650292 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="48092d6e-157f-4df3-828b-497746965c69" path="/var/lib/kubelet/pods/48092d6e-157f-4df3-828b-497746965c69/volumes" Oct 30 19:17:44.590442 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:44.590396 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:46.651716 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:46.651680 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2"] Oct 30 19:17:46.702273 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:46.702240 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2"] Oct 30 19:17:46.704518 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:46.704493 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g"] Oct 30 19:17:46.795783 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:46.795750 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:49.787175 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:49.787136 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:49.920224 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:49.920180 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" podUID="ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" containerName="main" containerID="cri-o://a16415535e32f16a18f7423509746f11c704a8a706b04e433e746ca0450af7ca" gracePeriod=45 Oct 30 19:17:50.125673 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:17:50.125620 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:17:50.184668 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.184640 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:17:50.318198 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.318120 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-dshm\") pod \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " Oct 30 19:17:50.318366 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.318248 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-model-cache\") pod \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " Oct 30 19:17:50.318366 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.318274 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-kube-api-access-qwxk6\" (UniqueName: \"kubernetes.io/projected/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-kube-api-access-qwxk6\") pod \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " Oct 30 19:17:50.318366 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.318295 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-home\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-home\") pod \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " Oct 30 19:17:50.318366 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.318357 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-tls-certs\" (UniqueName: \"kubernetes.io/secret/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-tls-certs\") pod \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " Oct 30 19:17:50.318625 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.318430 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-tmp-dir\") pod \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\" (UID: \"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2\") " Oct 30 19:17:50.319750 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.319226 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-tmp-dir" pod "ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" (UID: "ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:50.319750 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.319415 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-home" pod "ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" (UID: "ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:50.319750 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.319712 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-model-cache" pod "ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" (UID: "ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:50.320742 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.320712 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-dshm" pod "ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" (UID: "ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:17:50.321130 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.321113 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-kube-api-access-qwxk6" pod "ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" (UID: "ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2"). InnerVolumeSpecName "kube-api-access-qwxk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:17:50.322433 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.322406 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-tls-certs" pod "ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" (UID: "ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:17:50.420238 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.420205 5894 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-tmp-dir\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:50.420238 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.420235 5894 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-dshm\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:50.420238 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.420244 5894 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-model-cache\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:50.420451 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.420255 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwxk6\" (UniqueName: \"kubernetes.io/projected/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-kube-api-access-qwxk6\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:50.420451 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.420264 5894 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-home\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:50.420451 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.420273 5894 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2-tls-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:17:50.630325 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.630243 5894 generic.go:356] "Generic (PLEG): container finished" podID="ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" containerID="a16415535e32f16a18f7423509746f11c704a8a706b04e433e746ca0450af7ca" exitCode=0 Oct 30 19:17:50.630325 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.630291 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" Oct 30 19:17:50.630522 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.630318 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" event={"ID":"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2","Type":"ContainerDied","Data":"a16415535e32f16a18f7423509746f11c704a8a706b04e433e746ca0450af7ca"} Oct 30 19:17:50.630522 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.630365 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g" event={"ID":"ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2","Type":"ContainerDied","Data":"e96553520cb795f4257f2bfdf951ee53c7a3cc9907532541d158fd59bf63c494"} Oct 30 19:17:50.630522 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.630383 5894 scope.go:122] "RemoveContainer" containerID="a16415535e32f16a18f7423509746f11c704a8a706b04e433e746ca0450af7ca" Oct 30 19:17:50.640738 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.640720 5894 scope.go:122] "RemoveContainer" containerID="a16415535e32f16a18f7423509746f11c704a8a706b04e433e746ca0450af7ca" Oct 30 19:17:50.641028 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:17:50.641001 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a16415535e32f16a18f7423509746f11c704a8a706b04e433e746ca0450af7ca\": container with ID starting with a16415535e32f16a18f7423509746f11c704a8a706b04e433e746ca0450af7ca not found: ID does not exist" containerID="a16415535e32f16a18f7423509746f11c704a8a706b04e433e746ca0450af7ca" Oct 30 19:17:50.641083 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.641038 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a16415535e32f16a18f7423509746f11c704a8a706b04e433e746ca0450af7ca"} err="failed to get container status \"a16415535e32f16a18f7423509746f11c704a8a706b04e433e746ca0450af7ca\": rpc error: code = NotFound desc = could not find container \"a16415535e32f16a18f7423509746f11c704a8a706b04e433e746ca0450af7ca\": container with ID starting with a16415535e32f16a18f7423509746f11c704a8a706b04e433e746ca0450af7ca not found: ID does not exist" Oct 30 19:17:50.794307 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.794274 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g"] Oct 30 19:17:50.843843 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:50.843813 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-975cf5/distinct-2-w1-975cf5-kserve-b58c88d6b-4429g"] Oct 30 19:17:51.649767 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:51.649735 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" path="/var/lib/kubelet/pods/ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2/volumes" Oct 30 19:17:52.450900 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:52.450845 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:52.642355 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:52.642321 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:53.861124 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:53.861086 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:54.064988 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:54.064949 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:54.640092 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:54.640053 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:55.319000 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:55.318961 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:55.503397 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:55.503318 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:56.086273 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:56.086240 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:17:59.380508 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:17:59.380468 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:01.672649 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:01.672548 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" containerID="cri-o://7fb7221e4aae820f53bffa99bd0af35cc22ac02e91de796e862eb9462193ea2f" gracePeriod=45 Oct 30 19:18:01.723299 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:01.723259 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" containerID="cri-o://93ce532769e04006fc047067e33e920c2aa8976a517609e0f4672042f808ae06" gracePeriod=45 Oct 30 19:18:01.724363 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:01.724329 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" containerID="cri-o://b79d0b3a3aee505e588b2912cb773a8a17b81a37fc2c64e9265436646e84f531" gracePeriod=45 Oct 30 19:18:01.828237 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:01.828183 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:02.024341 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.023175 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:02.030649 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.030609 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:18:02.056979 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.056948 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 19:18:02.060965 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.060942 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 19:18:02.080529 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.080496 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-home\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-home\") pod \"b42c4711-cc79-4024-b2b9-c6294004c373\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " Oct 30 19:18:02.080737 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.080597 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-tmp-dir\") pod \"b42c4711-cc79-4024-b2b9-c6294004c373\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " Oct 30 19:18:02.080737 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.080647 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-dshm\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-dshm\") pod \"b42c4711-cc79-4024-b2b9-c6294004c373\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " Oct 30 19:18:02.080737 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.080678 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-tmp-dir\") pod \"4049821e-28c6-4718-9ed9-01101012c150\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " Oct 30 19:18:02.080737 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.080703 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-home\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-home\") pod \"4049821e-28c6-4718-9ed9-01101012c150\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " Oct 30 19:18:02.080972 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.080770 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-home\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-home\") pod \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " Oct 30 19:18:02.080972 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.080808 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-home" pod "b42c4711-cc79-4024-b2b9-c6294004c373" (UID: "b42c4711-cc79-4024-b2b9-c6294004c373"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:02.080972 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.080822 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-model-cache\") pod \"4049821e-28c6-4718-9ed9-01101012c150\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " Oct 30 19:18:02.080972 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.080877 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-tmp-dir\") pod \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " Oct 30 19:18:02.080972 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.080901 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-tmp-dir" pod "b42c4711-cc79-4024-b2b9-c6294004c373" (UID: "b42c4711-cc79-4024-b2b9-c6294004c373"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:02.080972 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.080961 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-model-cache\") pod \"b42c4711-cc79-4024-b2b9-c6294004c373\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " Oct 30 19:18:02.081327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.081013 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-tmp-dir" pod "4049821e-28c6-4718-9ed9-01101012c150" (UID: "4049821e-28c6-4718-9ed9-01101012c150"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:02.081327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.081037 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/b42c4711-cc79-4024-b2b9-c6294004c373-tls-certs\" (UniqueName: \"kubernetes.io/secret/b42c4711-cc79-4024-b2b9-c6294004c373-tls-certs\") pod \"b42c4711-cc79-4024-b2b9-c6294004c373\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " Oct 30 19:18:02.081327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.081074 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-dshm\") pod \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " Oct 30 19:18:02.081327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.081104 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-kube-api-access-b4bqd\" (UniqueName: \"kubernetes.io/projected/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-kube-api-access-b4bqd\") pod \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " Oct 30 19:18:02.081327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.081144 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-home" pod "ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" (UID: "ee442ed1-4b79-47ab-b0d9-00cc2b0d717a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:02.081327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.081162 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4049821e-28c6-4718-9ed9-01101012c150-tls-certs\" (UniqueName: \"kubernetes.io/secret/4049821e-28c6-4718-9ed9-01101012c150-tls-certs\") pod \"4049821e-28c6-4718-9ed9-01101012c150\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " Oct 30 19:18:02.081327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.081199 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-model-cache\") pod \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " Oct 30 19:18:02.081327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.081248 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4049821e-28c6-4718-9ed9-01101012c150-kube-api-access-4kht9\" (UniqueName: \"kubernetes.io/projected/4049821e-28c6-4718-9ed9-01101012c150-kube-api-access-4kht9\") pod \"4049821e-28c6-4718-9ed9-01101012c150\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " Oct 30 19:18:02.081327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.081262 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-model-cache" pod "4049821e-28c6-4718-9ed9-01101012c150" (UID: "4049821e-28c6-4718-9ed9-01101012c150"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:02.081327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.081274 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-tls-certs\" (UniqueName: \"kubernetes.io/secret/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-tls-certs\") pod \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\" (UID: \"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a\") " Oct 30 19:18:02.081327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.081310 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b42c4711-cc79-4024-b2b9-c6294004c373-kube-api-access-h7zkl\" (UniqueName: \"kubernetes.io/projected/b42c4711-cc79-4024-b2b9-c6294004c373-kube-api-access-h7zkl\") pod \"b42c4711-cc79-4024-b2b9-c6294004c373\" (UID: \"b42c4711-cc79-4024-b2b9-c6294004c373\") " Oct 30 19:18:02.081904 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.081337 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-dshm\") pod \"4049821e-28c6-4718-9ed9-01101012c150\" (UID: \"4049821e-28c6-4718-9ed9-01101012c150\") " Oct 30 19:18:02.081904 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.081359 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-home" pod "4049821e-28c6-4718-9ed9-01101012c150" (UID: "4049821e-28c6-4718-9ed9-01101012c150"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:02.081904 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.081515 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-model-cache" pod "b42c4711-cc79-4024-b2b9-c6294004c373" (UID: "b42c4711-cc79-4024-b2b9-c6294004c373"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:02.081904 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.081556 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-model-cache" pod "ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" (UID: "ee442ed1-4b79-47ab-b0d9-00cc2b0d717a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:02.081904 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.081866 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-tmp-dir" pod "ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" (UID: "ee442ed1-4b79-47ab-b0d9-00cc2b0d717a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:02.082301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.081955 5894 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-model-cache\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:02.082301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.081977 5894 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-home\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:02.082301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.081991 5894 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-tmp-dir\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:02.082301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.082005 5894 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-tmp-dir\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:02.082301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.082022 5894 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-home\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:02.082301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.082035 5894 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-home\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:02.082301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.082048 5894 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-model-cache\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:02.082301 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.082064 5894 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-model-cache\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:02.084480 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.084242 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b42c4711-cc79-4024-b2b9-c6294004c373-kube-api-access-h7zkl" pod "b42c4711-cc79-4024-b2b9-c6294004c373" (UID: "b42c4711-cc79-4024-b2b9-c6294004c373"). InnerVolumeSpecName "kube-api-access-h7zkl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:18:02.084635 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.084543 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b42c4711-cc79-4024-b2b9-c6294004c373-tls-certs" pod "b42c4711-cc79-4024-b2b9-c6294004c373" (UID: "b42c4711-cc79-4024-b2b9-c6294004c373"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:18:02.084635 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.084551 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-dshm" pod "ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" (UID: "ee442ed1-4b79-47ab-b0d9-00cc2b0d717a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:02.085371 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.085343 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4049821e-28c6-4718-9ed9-01101012c150-tls-certs" pod "4049821e-28c6-4718-9ed9-01101012c150" (UID: "4049821e-28c6-4718-9ed9-01101012c150"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:18:02.085483 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.085425 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-dshm" pod "4049821e-28c6-4718-9ed9-01101012c150" (UID: "4049821e-28c6-4718-9ed9-01101012c150"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:02.085483 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.085439 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-tls-certs" pod "ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" (UID: "ee442ed1-4b79-47ab-b0d9-00cc2b0d717a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:18:02.085483 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.085447 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-dshm" pod "b42c4711-cc79-4024-b2b9-c6294004c373" (UID: "b42c4711-cc79-4024-b2b9-c6294004c373"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:02.085630 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.085486 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-kube-api-access-b4bqd" pod "ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" (UID: "ee442ed1-4b79-47ab-b0d9-00cc2b0d717a"). InnerVolumeSpecName "kube-api-access-b4bqd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:18:02.086391 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.086374 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4049821e-28c6-4718-9ed9-01101012c150-kube-api-access-4kht9" pod "4049821e-28c6-4718-9ed9-01101012c150" (UID: "4049821e-28c6-4718-9ed9-01101012c150"). InnerVolumeSpecName "kube-api-access-4kht9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:18:02.183047 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.183009 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4kht9\" (UniqueName: \"kubernetes.io/projected/4049821e-28c6-4718-9ed9-01101012c150-kube-api-access-4kht9\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:02.183047 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.183047 5894 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-tls-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:02.183300 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.183061 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7zkl\" (UniqueName: \"kubernetes.io/projected/b42c4711-cc79-4024-b2b9-c6294004c373-kube-api-access-h7zkl\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:02.183300 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.183076 5894 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4049821e-28c6-4718-9ed9-01101012c150-dshm\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:02.183300 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.183088 5894 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b42c4711-cc79-4024-b2b9-c6294004c373-dshm\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:02.183300 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.183100 5894 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-tmp-dir\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:02.183300 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.183111 5894 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b42c4711-cc79-4024-b2b9-c6294004c373-tls-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:02.183300 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.183123 5894 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-dshm\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:02.183300 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.183135 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4bqd\" (UniqueName: \"kubernetes.io/projected/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a-kube-api-access-b4bqd\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:02.183300 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.183149 5894 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4049821e-28c6-4718-9ed9-01101012c150-tls-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:02.715090 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.714224 5894 generic.go:356] "Generic (PLEG): container finished" podID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerID="93ce532769e04006fc047067e33e920c2aa8976a517609e0f4672042f808ae06" exitCode=0 Oct 30 19:18:02.715090 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.714422 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" event={"ID":"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a","Type":"ContainerDied","Data":"93ce532769e04006fc047067e33e920c2aa8976a517609e0f4672042f808ae06"} Oct 30 19:18:02.715090 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.715041 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" Oct 30 19:18:02.715707 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.715322 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2" event={"ID":"ee442ed1-4b79-47ab-b0d9-00cc2b0d717a","Type":"ContainerDied","Data":"c533e0fd2e279f627b124e06d0c5251897c6cf6a712fc74ce9410a91818ac597"} Oct 30 19:18:02.715707 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.715370 5894 scope.go:122] "RemoveContainer" containerID="93ce532769e04006fc047067e33e920c2aa8976a517609e0f4672042f808ae06" Oct 30 19:18:02.718751 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.718463 5894 generic.go:356] "Generic (PLEG): container finished" podID="b42c4711-cc79-4024-b2b9-c6294004c373" containerID="7fb7221e4aae820f53bffa99bd0af35cc22ac02e91de796e862eb9462193ea2f" exitCode=0 Oct 30 19:18:02.718751 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.718649 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" Oct 30 19:18:02.718903 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.718873 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" event={"ID":"b42c4711-cc79-4024-b2b9-c6294004c373","Type":"ContainerDied","Data":"7fb7221e4aae820f53bffa99bd0af35cc22ac02e91de796e862eb9462193ea2f"} Oct 30 19:18:02.719033 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.718920 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2" event={"ID":"b42c4711-cc79-4024-b2b9-c6294004c373","Type":"ContainerDied","Data":"fc25bffcfdb5fea4370b957147e5418448b18491a0fb23c8339c8476a5acb9df"} Oct 30 19:18:02.722385 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.722356 5894 generic.go:356] "Generic (PLEG): container finished" podID="4049821e-28c6-4718-9ed9-01101012c150" containerID="b79d0b3a3aee505e588b2912cb773a8a17b81a37fc2c64e9265436646e84f531" exitCode=0 Oct 30 19:18:02.722503 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.722439 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" event={"ID":"4049821e-28c6-4718-9ed9-01101012c150","Type":"ContainerDied","Data":"b79d0b3a3aee505e588b2912cb773a8a17b81a37fc2c64e9265436646e84f531"} Oct 30 19:18:02.722503 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.722465 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" event={"ID":"4049821e-28c6-4718-9ed9-01101012c150","Type":"ContainerDied","Data":"490ab828b6f5c4ae9598ccc51e6e4aefb148788907a4ce3c474fdc05758470c8"} Oct 30 19:18:02.722656 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.722522 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g" Oct 30 19:18:02.731656 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.731636 5894 scope.go:122] "RemoveContainer" containerID="93ce532769e04006fc047067e33e920c2aa8976a517609e0f4672042f808ae06" Oct 30 19:18:02.731978 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:18:02.731959 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93ce532769e04006fc047067e33e920c2aa8976a517609e0f4672042f808ae06\": container with ID starting with 93ce532769e04006fc047067e33e920c2aa8976a517609e0f4672042f808ae06 not found: ID does not exist" containerID="93ce532769e04006fc047067e33e920c2aa8976a517609e0f4672042f808ae06" Oct 30 19:18:02.732022 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.731987 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93ce532769e04006fc047067e33e920c2aa8976a517609e0f4672042f808ae06"} err="failed to get container status \"93ce532769e04006fc047067e33e920c2aa8976a517609e0f4672042f808ae06\": rpc error: code = NotFound desc = could not find container \"93ce532769e04006fc047067e33e920c2aa8976a517609e0f4672042f808ae06\": container with ID starting with 93ce532769e04006fc047067e33e920c2aa8976a517609e0f4672042f808ae06 not found: ID does not exist" Oct 30 19:18:02.732022 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.732005 5894 scope.go:122] "RemoveContainer" containerID="7fb7221e4aae820f53bffa99bd0af35cc22ac02e91de796e862eb9462193ea2f" Oct 30 19:18:02.741688 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.741670 5894 scope.go:122] "RemoveContainer" containerID="7fb7221e4aae820f53bffa99bd0af35cc22ac02e91de796e862eb9462193ea2f" Oct 30 19:18:02.741966 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:18:02.741938 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7fb7221e4aae820f53bffa99bd0af35cc22ac02e91de796e862eb9462193ea2f\": container with ID starting with 7fb7221e4aae820f53bffa99bd0af35cc22ac02e91de796e862eb9462193ea2f not found: ID does not exist" containerID="7fb7221e4aae820f53bffa99bd0af35cc22ac02e91de796e862eb9462193ea2f" Oct 30 19:18:02.742019 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.741977 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7fb7221e4aae820f53bffa99bd0af35cc22ac02e91de796e862eb9462193ea2f"} err="failed to get container status \"7fb7221e4aae820f53bffa99bd0af35cc22ac02e91de796e862eb9462193ea2f\": rpc error: code = NotFound desc = could not find container \"7fb7221e4aae820f53bffa99bd0af35cc22ac02e91de796e862eb9462193ea2f\": container with ID starting with 7fb7221e4aae820f53bffa99bd0af35cc22ac02e91de796e862eb9462193ea2f not found: ID does not exist" Oct 30 19:18:02.742019 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.742001 5894 scope.go:122] "RemoveContainer" containerID="b79d0b3a3aee505e588b2912cb773a8a17b81a37fc2c64e9265436646e84f531" Oct 30 19:18:02.753999 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.753978 5894 scope.go:122] "RemoveContainer" containerID="b79d0b3a3aee505e588b2912cb773a8a17b81a37fc2c64e9265436646e84f531" Oct 30 19:18:02.754373 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:18:02.754350 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b79d0b3a3aee505e588b2912cb773a8a17b81a37fc2c64e9265436646e84f531\": container with ID starting with b79d0b3a3aee505e588b2912cb773a8a17b81a37fc2c64e9265436646e84f531 not found: ID does not exist" containerID="b79d0b3a3aee505e588b2912cb773a8a17b81a37fc2c64e9265436646e84f531" Oct 30 19:18:02.754433 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.754384 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b79d0b3a3aee505e588b2912cb773a8a17b81a37fc2c64e9265436646e84f531"} err="failed to get container status \"b79d0b3a3aee505e588b2912cb773a8a17b81a37fc2c64e9265436646e84f531\": rpc error: code = NotFound desc = could not find container \"b79d0b3a3aee505e588b2912cb773a8a17b81a37fc2c64e9265436646e84f531\": container with ID starting with b79d0b3a3aee505e588b2912cb773a8a17b81a37fc2c64e9265436646e84f531 not found: ID does not exist" Oct 30 19:18:02.878814 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.878773 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g"] Oct 30 19:18:02.928556 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.928523 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-1290a6/simulator-w4-1290a6-kserve-687975dc7b-qls8g"] Oct 30 19:18:02.993722 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.993639 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2"] Oct 30 19:18:02.999376 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:02.999349 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-1290a6/premium-w4-1290a6-kserve-86cf84b59c-rhgn2"] Oct 30 19:18:03.117465 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:03.117432 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2"] Oct 30 19:18:03.127857 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:03.127825 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-1290a6/distinct-2-w4-1290a6-kserve-56748dbb7-bsqk2"] Oct 30 19:18:03.391472 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:03.391436 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:03.651996 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:03.651917 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4049821e-28c6-4718-9ed9-01101012c150" path="/var/lib/kubelet/pods/4049821e-28c6-4718-9ed9-01101012c150/volumes" Oct 30 19:18:03.652388 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:03.652374 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" path="/var/lib/kubelet/pods/b42c4711-cc79-4024-b2b9-c6294004c373/volumes" Oct 30 19:18:03.652804 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:03.652791 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" path="/var/lib/kubelet/pods/ee442ed1-4b79-47ab-b0d9-00cc2b0d717a/volumes" Oct 30 19:18:03.934353 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:03.934278 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:04.311748 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:04.309238 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:08.987475 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:08.987443 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:09.654446 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:09.654407 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:11.811534 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:11.811499 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:11.995033 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:11.994996 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:12.847444 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:12.847412 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:13.133981 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:13.133898 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:14.722567 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:14.722536 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:14.980367 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:14.980290 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:15.169496 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:15.169462 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:15.676888 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:15.676854 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:18.376610 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:18.376563 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:20.118393 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:18:20.118350 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:18:20.369284 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:20.369170 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:21.700982 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:21.700946 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:21.895314 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:21.895284 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:22.167934 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:22.167897 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:23.390250 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:23.390215 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:23.573234 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:23.573196 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:24.622595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:24.621982 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:25.399413 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:25.399383 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:25.625835 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:25.625801 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:30.425490 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:30.425457 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:30.599263 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:30.599231 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:30.901255 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:30.891123 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:34.227850 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:34.227817 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj"] Oct 30 19:18:34.566210 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:34.566108 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:34.748432 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:34.748398 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:34.814713 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:34.814678 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5"] Oct 30 19:18:34.814918 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:34.814893 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" podUID="df3270ce-707c-45f4-a0d9-3dfd137fbdfc" containerName="maas-api" containerID="cri-o://71c5374104491265deb4936ab6d2f2cc64b8faddcb989be8c71f4df8ab830edc" gracePeriod=30 Oct 30 19:18:34.954815 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:34.954784 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:34.960690 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:34.960647 5894 generic.go:356] "Generic (PLEG): container finished" podID="df3270ce-707c-45f4-a0d9-3dfd137fbdfc" containerID="71c5374104491265deb4936ab6d2f2cc64b8faddcb989be8c71f4df8ab830edc" exitCode=0 Oct 30 19:18:34.960859 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:34.960725 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" event={"ID":"df3270ce-707c-45f4-a0d9-3dfd137fbdfc","Type":"ContainerDied","Data":"71c5374104491265deb4936ab6d2f2cc64b8faddcb989be8c71f4df8ab830edc"} Oct 30 19:18:35.169512 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:35.169486 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" Oct 30 19:18:35.289209 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:35.288052 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-openshift-service-ca\") pod \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\" (UID: \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\") " Oct 30 19:18:35.289209 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:35.288111 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-maas-api-tls\") pod \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\" (UID: \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\") " Oct 30 19:18:35.289209 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:35.288298 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-metrics-certs\" (UniqueName: \"kubernetes.io/secret/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-metrics-certs\") pod \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\" (UID: \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\") " Oct 30 19:18:35.289209 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:35.288347 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-kube-api-access-9lw7c\" (UniqueName: \"kubernetes.io/projected/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-kube-api-access-9lw7c\") pod \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\" (UID: \"df3270ce-707c-45f4-a0d9-3dfd137fbdfc\") " Oct 30 19:18:35.290470 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:35.290415 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-openshift-service-ca" pod "df3270ce-707c-45f4-a0d9-3dfd137fbdfc" (UID: "df3270ce-707c-45f4-a0d9-3dfd137fbdfc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:18:35.294007 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:35.293979 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-kube-api-access-9lw7c" pod "df3270ce-707c-45f4-a0d9-3dfd137fbdfc" (UID: "df3270ce-707c-45f4-a0d9-3dfd137fbdfc"). InnerVolumeSpecName "kube-api-access-9lw7c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:18:35.297980 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:35.294237 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-metrics-certs" pod "df3270ce-707c-45f4-a0d9-3dfd137fbdfc" (UID: "df3270ce-707c-45f4-a0d9-3dfd137fbdfc"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:18:35.315527 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:35.315138 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-maas-api-tls" pod "df3270ce-707c-45f4-a0d9-3dfd137fbdfc" (UID: "df3270ce-707c-45f4-a0d9-3dfd137fbdfc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:18:35.389866 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:35.389836 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-maas-api-tls\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:35.389866 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:35.389864 5894 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-metrics-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:35.389866 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:35.389874 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lw7c\" (UniqueName: \"kubernetes.io/projected/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-kube-api-access-9lw7c\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:35.390088 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:35.389885 5894 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df3270ce-707c-45f4-a0d9-3dfd137fbdfc-openshift-service-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:35.967671 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:35.967590 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" Oct 30 19:18:35.967852 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:35.967592 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5" event={"ID":"df3270ce-707c-45f4-a0d9-3dfd137fbdfc","Type":"ContainerDied","Data":"13f023457d67d734dbf02aadad509ef49f468d074e2dbb7e47b865a9e6f397b9"} Oct 30 19:18:35.967852 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:35.967736 5894 scope.go:122] "RemoveContainer" containerID="71c5374104491265deb4936ab6d2f2cc64b8faddcb989be8c71f4df8ab830edc" Oct 30 19:18:36.118660 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:36.118624 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:36.170340 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:36.170305 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5"] Oct 30 19:18:36.225804 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:36.225723 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w6-41db7c-544979c745-9cbv5"] Oct 30 19:18:36.493638 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:36.493552 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:36.769493 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:36.769412 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:37.398886 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:37.398848 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:37.650798 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:37.650725 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="df3270ce-707c-45f4-a0d9-3dfd137fbdfc" path="/var/lib/kubelet/pods/df3270ce-707c-45f4-a0d9-3dfd137fbdfc/volumes" Oct 30 19:18:38.004659 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:38.001491 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:39.228313 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:39.228266 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj" podUID="3df940b4-b684-4a85-ae0c-2fe4ffe51849" containerName="payload-pre-processing" containerID="cri-o://59402d10975d07677e34bf3f5b0248eea4df1908f48f0b1ae3f3b76b0a4f7a70" gracePeriod=25 Oct 30 19:18:39.505084 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:39.505061 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj" Oct 30 19:18:39.567921 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:39.567892 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3df940b4-b684-4a85-ae0c-2fe4ffe51849-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3df940b4-b684-4a85-ae0c-2fe4ffe51849-plugins-config-volume\") pod \"3df940b4-b684-4a85-ae0c-2fe4ffe51849\" (UID: \"3df940b4-b684-4a85-ae0c-2fe4ffe51849\") " Oct 30 19:18:39.568102 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:39.567940 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3df940b4-b684-4a85-ae0c-2fe4ffe51849-kube-api-access-kg2xw\" (UniqueName: \"kubernetes.io/projected/3df940b4-b684-4a85-ae0c-2fe4ffe51849-kube-api-access-kg2xw\") pod \"3df940b4-b684-4a85-ae0c-2fe4ffe51849\" (UID: \"3df940b4-b684-4a85-ae0c-2fe4ffe51849\") " Oct 30 19:18:39.568282 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:39.568255 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3df940b4-b684-4a85-ae0c-2fe4ffe51849-plugins-config-volume" pod "3df940b4-b684-4a85-ae0c-2fe4ffe51849" (UID: "3df940b4-b684-4a85-ae0c-2fe4ffe51849"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:18:39.570137 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:39.570117 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3df940b4-b684-4a85-ae0c-2fe4ffe51849-kube-api-access-kg2xw" pod "3df940b4-b684-4a85-ae0c-2fe4ffe51849" (UID: "3df940b4-b684-4a85-ae0c-2fe4ffe51849"). InnerVolumeSpecName "kube-api-access-kg2xw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:18:39.669931 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:39.669897 5894 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3df940b4-b684-4a85-ae0c-2fe4ffe51849-plugins-config-volume\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:39.670056 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:39.669936 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kg2xw\" (UniqueName: \"kubernetes.io/projected/3df940b4-b684-4a85-ae0c-2fe4ffe51849-kube-api-access-kg2xw\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:40.024773 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:40.024735 5894 generic.go:356] "Generic (PLEG): container finished" podID="3df940b4-b684-4a85-ae0c-2fe4ffe51849" containerID="59402d10975d07677e34bf3f5b0248eea4df1908f48f0b1ae3f3b76b0a4f7a70" exitCode=0 Oct 30 19:18:40.024940 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:40.024784 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj" Oct 30 19:18:40.024940 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:40.024823 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj" event={"ID":"3df940b4-b684-4a85-ae0c-2fe4ffe51849","Type":"ContainerDied","Data":"59402d10975d07677e34bf3f5b0248eea4df1908f48f0b1ae3f3b76b0a4f7a70"} Oct 30 19:18:40.024940 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:40.024881 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj" event={"ID":"3df940b4-b684-4a85-ae0c-2fe4ffe51849","Type":"ContainerDied","Data":"d666849e4c7313ff207bf3ff21e39921a5d7ff3620d4675fa0bbc150b104f6b1"} Oct 30 19:18:40.024940 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:40.024905 5894 scope.go:122] "RemoveContainer" containerID="59402d10975d07677e34bf3f5b0248eea4df1908f48f0b1ae3f3b76b0a4f7a70" Oct 30 19:18:40.034367 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:40.034349 5894 scope.go:122] "RemoveContainer" containerID="59402d10975d07677e34bf3f5b0248eea4df1908f48f0b1ae3f3b76b0a4f7a70" Oct 30 19:18:40.034685 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:18:40.034666 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59402d10975d07677e34bf3f5b0248eea4df1908f48f0b1ae3f3b76b0a4f7a70\": container with ID starting with 59402d10975d07677e34bf3f5b0248eea4df1908f48f0b1ae3f3b76b0a4f7a70 not found: ID does not exist" containerID="59402d10975d07677e34bf3f5b0248eea4df1908f48f0b1ae3f3b76b0a4f7a70" Oct 30 19:18:40.034764 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:40.034695 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59402d10975d07677e34bf3f5b0248eea4df1908f48f0b1ae3f3b76b0a4f7a70"} err="failed to get container status \"59402d10975d07677e34bf3f5b0248eea4df1908f48f0b1ae3f3b76b0a4f7a70\": rpc error: code = NotFound desc = could not find container \"59402d10975d07677e34bf3f5b0248eea4df1908f48f0b1ae3f3b76b0a4f7a70\": container with ID starting with 59402d10975d07677e34bf3f5b0248eea4df1908f48f0b1ae3f3b76b0a4f7a70 not found: ID does not exist" Oct 30 19:18:40.190890 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:40.190856 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj"] Oct 30 19:18:40.240815 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:40.240784 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w3-36d676-ff545855f-cltzj"] Oct 30 19:18:41.650074 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:41.650036 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3df940b4-b684-4a85-ae0c-2fe4ffe51849" path="/var/lib/kubelet/pods/3df940b4-b684-4a85-ae0c-2fe4ffe51849/volumes" Oct 30 19:18:43.062601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:43.058403 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:43.615943 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:43.615909 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:43.797269 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:43.797229 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:43.900613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:43.900554 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs"] Oct 30 19:18:43.900882 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:43.900855 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" podUID="e4b4e21b-a660-4b77-869c-64ec2a9996be" containerName="istio-proxy" containerID="cri-o://dc633cfa9302d94206772134950af8f1055edcabf0ee8a9813db39b5cec4f282" gracePeriod=30 Oct 30 19:18:43.996533 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:43.989402 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:45.333709 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:45.333667 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:45.558911 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:45.558854 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:45.748503 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:45.748454 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:46.013868 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:46.013789 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d"] Oct 30 19:18:46.014050 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:46.014024 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" podUID="a1d0e8a5-75ba-4e8d-9168-384428b30c4c" containerName="istio-proxy" containerID="cri-o://5df39f9c3205bd39046421c940f23a02837a64781b9c56d8efc7e827617eacbf" gracePeriod=30 Oct 30 19:18:46.028404 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:46.028376 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d"] Oct 30 19:18:46.028615 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:46.028589 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" podUID="1a6b1680-232d-4c50-beee-d3967a408fd4" containerName="istio-proxy" containerID="cri-o://03975e70e8168300f82971dfc7ed26bd583c25fd8d0a5b872ba6331498f90188" gracePeriod=30 Oct 30 19:18:46.057078 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:46.057044 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5"] Oct 30 19:18:46.057303 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:46.057279 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" podUID="1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" containerName="istio-proxy" containerID="cri-o://551910f8f6104ea23e3ee182d2863d1fd37b505e613e49aa1f8216fc8da79a07" gracePeriod=30 Oct 30 19:18:46.060707 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:46.060679 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:46.062302 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:46.062280 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27"] Oct 30 19:18:46.062476 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:46.062456 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" podUID="0bff1115-e48c-4282-868e-4a572b4c12cf" containerName="istio-proxy" containerID="cri-o://d9963cdea518f4510e4e475f270bbab446724f750cc6c9e65e9ec19f58b81f70" gracePeriod=30 Oct 30 19:18:46.927557 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:46.927520 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:47.130311 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:47.130274 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:48.192218 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:48.190534 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:48.393802 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:48.393765 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:49.155473 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.155441 5894 generic.go:356] "Generic (PLEG): container finished" podID="e4b4e21b-a660-4b77-869c-64ec2a9996be" containerID="dc633cfa9302d94206772134950af8f1055edcabf0ee8a9813db39b5cec4f282" exitCode=0 Oct 30 19:18:49.155620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.155510 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" event={"ID":"e4b4e21b-a660-4b77-869c-64ec2a9996be","Type":"ContainerDied","Data":"dc633cfa9302d94206772134950af8f1055edcabf0ee8a9813db39b5cec4f282"} Oct 30 19:18:49.155620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.155558 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" event={"ID":"e4b4e21b-a660-4b77-869c-64ec2a9996be","Type":"ContainerDied","Data":"d0ab90d0a96fecbb1c09f45dcc703383e9ee7e79bd7e48e19f5000fde95c5542"} Oct 30 19:18:49.155620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.155592 5894 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d0ab90d0a96fecbb1c09f45dcc703383e9ee7e79bd7e48e19f5000fde95c5542" Oct 30 19:18:49.163505 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.163458 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 19:18:49.305045 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.305013 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-podinfo\") pod \"e4b4e21b-a660-4b77-869c-64ec2a9996be\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " Oct 30 19:18:49.305444 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.305067 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-envoy\") pod \"e4b4e21b-a660-4b77-869c-64ec2a9996be\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " Oct 30 19:18:49.305444 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.305099 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-credential-socket\") pod \"e4b4e21b-a660-4b77-869c-64ec2a9996be\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " Oct 30 19:18:49.305444 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.305141 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-token\" (UniqueName: \"kubernetes.io/projected/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-token\") pod \"e4b4e21b-a660-4b77-869c-64ec2a9996be\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " Oct 30 19:18:49.305444 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.305238 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-data\") pod \"e4b4e21b-a660-4b77-869c-64ec2a9996be\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " Oct 30 19:18:49.305444 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.305284 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e4b4e21b-a660-4b77-869c-64ec2a9996be-kube-api-access-jbb78\" (UniqueName: \"kubernetes.io/projected/e4b4e21b-a660-4b77-869c-64ec2a9996be-kube-api-access-jbb78\") pod \"e4b4e21b-a660-4b77-869c-64ec2a9996be\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " Oct 30 19:18:49.305444 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.305347 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-workload-socket\") pod \"e4b4e21b-a660-4b77-869c-64ec2a9996be\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " Oct 30 19:18:49.305444 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.305378 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/e4b4e21b-a660-4b77-869c-64ec2a9996be-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4b4e21b-a660-4b77-869c-64ec2a9996be-istiod-ca-cert\") pod \"e4b4e21b-a660-4b77-869c-64ec2a9996be\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " Oct 30 19:18:49.305444 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.305419 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-credential-socket" pod "e4b4e21b-a660-4b77-869c-64ec2a9996be" (UID: "e4b4e21b-a660-4b77-869c-64ec2a9996be"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:49.306222 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.305452 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-workload-certs\") pod \"e4b4e21b-a660-4b77-869c-64ec2a9996be\" (UID: \"e4b4e21b-a660-4b77-869c-64ec2a9996be\") " Oct 30 19:18:49.306222 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.305632 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-data" pod "e4b4e21b-a660-4b77-869c-64ec2a9996be" (UID: "e4b4e21b-a660-4b77-869c-64ec2a9996be"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:49.306222 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.305659 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-workload-socket" pod "e4b4e21b-a660-4b77-869c-64ec2a9996be" (UID: "e4b4e21b-a660-4b77-869c-64ec2a9996be"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:49.306222 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.305854 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-workload-certs" pod "e4b4e21b-a660-4b77-869c-64ec2a9996be" (UID: "e4b4e21b-a660-4b77-869c-64ec2a9996be"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:49.306222 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.306001 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:49.306222 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.306020 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4b4e21b-a660-4b77-869c-64ec2a9996be-istiod-ca-cert" pod "e4b4e21b-a660-4b77-869c-64ec2a9996be" (UID: "e4b4e21b-a660-4b77-869c-64ec2a9996be"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:18:49.306222 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.306026 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:49.306222 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.306065 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:49.306222 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.306083 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:49.307478 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.307449 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-podinfo" pod "e4b4e21b-a660-4b77-869c-64ec2a9996be" (UID: "e4b4e21b-a660-4b77-869c-64ec2a9996be"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:18:49.307701 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.307677 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-token" pod "e4b4e21b-a660-4b77-869c-64ec2a9996be" (UID: "e4b4e21b-a660-4b77-869c-64ec2a9996be"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:18:49.307813 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.307792 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4b4e21b-a660-4b77-869c-64ec2a9996be-kube-api-access-jbb78" pod "e4b4e21b-a660-4b77-869c-64ec2a9996be" (UID: "e4b4e21b-a660-4b77-869c-64ec2a9996be"). InnerVolumeSpecName "kube-api-access-jbb78". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:18:49.307899 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.307814 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-envoy" pod "e4b4e21b-a660-4b77-869c-64ec2a9996be" (UID: "e4b4e21b-a660-4b77-869c-64ec2a9996be"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:49.407414 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.407376 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbb78\" (UniqueName: \"kubernetes.io/projected/e4b4e21b-a660-4b77-869c-64ec2a9996be-kube-api-access-jbb78\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:49.407595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.407419 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4b4e21b-a660-4b77-869c-64ec2a9996be-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:49.407595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.407434 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:49.407595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.407449 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:49.407595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:49.407462 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4b4e21b-a660-4b77-869c-64ec2a9996be-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:50.117601 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:18:50.117504 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:18:50.160945 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:50.160918 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs" Oct 30 19:18:50.268010 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:50.267980 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 19:18:50.291299 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:50.291276 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 19:18:50.351028 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:50.350995 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs"] Oct 30 19:18:50.400483 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:50.400449 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w6-41db7c-openshift-default-b679f98c9-cnkvs"] Oct 30 19:18:51.168776 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.168716 5894 generic.go:356] "Generic (PLEG): container finished" podID="a1d0e8a5-75ba-4e8d-9168-384428b30c4c" containerID="5df39f9c3205bd39046421c940f23a02837a64781b9c56d8efc7e827617eacbf" exitCode=0 Oct 30 19:18:51.168856 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.168791 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" event={"ID":"a1d0e8a5-75ba-4e8d-9168-384428b30c4c","Type":"ContainerDied","Data":"5df39f9c3205bd39046421c940f23a02837a64781b9c56d8efc7e827617eacbf"} Oct 30 19:18:51.170934 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.170906 5894 generic.go:356] "Generic (PLEG): container finished" podID="1a6b1680-232d-4c50-beee-d3967a408fd4" containerID="03975e70e8168300f82971dfc7ed26bd583c25fd8d0a5b872ba6331498f90188" exitCode=0 Oct 30 19:18:51.171064 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.170961 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" event={"ID":"1a6b1680-232d-4c50-beee-d3967a408fd4","Type":"ContainerDied","Data":"03975e70e8168300f82971dfc7ed26bd583c25fd8d0a5b872ba6331498f90188"} Oct 30 19:18:51.349180 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.349153 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:18:51.389770 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.389745 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:18:51.408440 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.408414 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:18:51.421023 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.421004 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:18:51.529286 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.529257 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-workload-certs\") pod \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " Oct 30 19:18:51.529429 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.529292 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-token\" (UniqueName: \"kubernetes.io/projected/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-token\") pod \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " Oct 30 19:18:51.529429 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.529330 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-credential-socket\") pod \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " Oct 30 19:18:51.529429 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.529361 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istiod-ca-cert\") pod \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " Oct 30 19:18:51.529646 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.529441 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-data\") pod \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " Oct 30 19:18:51.529646 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.529609 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-workload-certs" pod "a1d0e8a5-75ba-4e8d-9168-384428b30c4c" (UID: "a1d0e8a5-75ba-4e8d-9168-384428b30c4c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.529767 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.529700 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-workload-certs\") pod \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " Oct 30 19:18:51.529767 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.529701 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-credential-socket" pod "a1d0e8a5-75ba-4e8d-9168-384428b30c4c" (UID: "a1d0e8a5-75ba-4e8d-9168-384428b30c4c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.529767 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.529711 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-data" pod "1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" (UID: "1d4f4ad1-2be4-4c2a-8614-1b71cd7365da"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.529924 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.529768 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-envoy\") pod \"1a6b1680-232d-4c50-beee-d3967a408fd4\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " Oct 30 19:18:51.529924 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.529815 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-credential-socket\") pod \"0bff1115-e48c-4282-868e-4a572b4c12cf\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " Oct 30 19:18:51.529924 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.529849 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-kube-api-access-jbpx9\" (UniqueName: \"kubernetes.io/projected/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-kube-api-access-jbpx9\") pod \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " Oct 30 19:18:51.529924 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.529893 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-workload-certs" pod "1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" (UID: "1d4f4ad1-2be4-4c2a-8614-1b71cd7365da"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.529924 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.529894 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-envoy\") pod \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " Oct 30 19:18:51.530182 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.529943 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-workload-socket\") pod \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " Oct 30 19:18:51.530182 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.529995 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a6b1680-232d-4c50-beee-d3967a408fd4-kube-api-access-8jw4c\" (UniqueName: \"kubernetes.io/projected/1a6b1680-232d-4c50-beee-d3967a408fd4-kube-api-access-8jw4c\") pod \"1a6b1680-232d-4c50-beee-d3967a408fd4\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " Oct 30 19:18:51.530182 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530025 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-envoy\") pod \"0bff1115-e48c-4282-868e-4a572b4c12cf\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " Oct 30 19:18:51.530182 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530081 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-data\") pod \"0bff1115-e48c-4282-868e-4a572b4c12cf\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " Oct 30 19:18:51.530182 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530104 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-credential-socket" pod "0bff1115-e48c-4282-868e-4a572b4c12cf" (UID: "0bff1115-e48c-4282-868e-4a572b4c12cf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.530182 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530113 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-envoy\") pod \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " Oct 30 19:18:51.530505 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530197 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-workload-certs\") pod \"0bff1115-e48c-4282-868e-4a572b4c12cf\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " Oct 30 19:18:51.530505 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530220 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istiod-ca-cert" pod "a1d0e8a5-75ba-4e8d-9168-384428b30c4c" (UID: "a1d0e8a5-75ba-4e8d-9168-384428b30c4c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:18:51.530505 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530228 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-workload-socket\") pod \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " Oct 30 19:18:51.530505 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530284 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istiod-ca-cert\") pod \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " Oct 30 19:18:51.530505 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530334 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1a6b1680-232d-4c50-beee-d3967a408fd4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a6b1680-232d-4c50-beee-d3967a408fd4-istiod-ca-cert\") pod \"1a6b1680-232d-4c50-beee-d3967a408fd4\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " Oct 30 19:18:51.530505 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530412 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-data\") pod \"1a6b1680-232d-4c50-beee-d3967a408fd4\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " Oct 30 19:18:51.530505 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530455 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-token\" (UniqueName: \"kubernetes.io/projected/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-token\") pod \"1a6b1680-232d-4c50-beee-d3967a408fd4\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " Oct 30 19:18:51.530505 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530498 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-workload-socket" pod "a1d0e8a5-75ba-4e8d-9168-384428b30c4c" (UID: "a1d0e8a5-75ba-4e8d-9168-384428b30c4c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.531009 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530518 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-podinfo\") pod \"1a6b1680-232d-4c50-beee-d3967a408fd4\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " Oct 30 19:18:51.531009 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530607 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-token\" (UniqueName: \"kubernetes.io/projected/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-token\") pod \"0bff1115-e48c-4282-868e-4a572b4c12cf\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " Oct 30 19:18:51.531009 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530655 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/0bff1115-e48c-4282-868e-4a572b4c12cf-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bff1115-e48c-4282-868e-4a572b4c12cf-istiod-ca-cert\") pod \"0bff1115-e48c-4282-868e-4a572b4c12cf\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " Oct 30 19:18:51.531009 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530700 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-workload-certs\") pod \"1a6b1680-232d-4c50-beee-d3967a408fd4\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " Oct 30 19:18:51.531009 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530749 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/0bff1115-e48c-4282-868e-4a572b4c12cf-kube-api-access-p6xdq\" (UniqueName: \"kubernetes.io/projected/0bff1115-e48c-4282-868e-4a572b4c12cf-kube-api-access-p6xdq\") pod \"0bff1115-e48c-4282-868e-4a572b4c12cf\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " Oct 30 19:18:51.531009 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530776 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-podinfo\") pod \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " Oct 30 19:18:51.531009 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530830 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-podinfo\") pod \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " Oct 30 19:18:51.531009 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530908 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-workload-socket\") pod \"0bff1115-e48c-4282-868e-4a572b4c12cf\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " Oct 30 19:18:51.531009 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.530938 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-token\" (UniqueName: \"kubernetes.io/projected/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-token\") pod \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " Oct 30 19:18:51.531009 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.531004 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-credential-socket\") pod \"1a6b1680-232d-4c50-beee-d3967a408fd4\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " Oct 30 19:18:51.531541 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.531053 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-podinfo\") pod \"0bff1115-e48c-4282-868e-4a572b4c12cf\" (UID: \"0bff1115-e48c-4282-868e-4a572b4c12cf\") " Oct 30 19:18:51.531541 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.531126 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-credential-socket\") pod \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " Oct 30 19:18:51.531541 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.531170 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-data\") pod \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\" (UID: \"a1d0e8a5-75ba-4e8d-9168-384428b30c4c\") " Oct 30 19:18:51.531541 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.531454 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-data" pod "0bff1115-e48c-4282-868e-4a572b4c12cf" (UID: "0bff1115-e48c-4282-868e-4a572b4c12cf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.531833 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.531806 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-workload-socket\") pod \"1a6b1680-232d-4c50-beee-d3967a408fd4\" (UID: \"1a6b1680-232d-4c50-beee-d3967a408fd4\") " Oct 30 19:18:51.531905 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.531864 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-kube-api-access-nrmql\" (UniqueName: \"kubernetes.io/projected/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-kube-api-access-nrmql\") pod \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\" (UID: \"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da\") " Oct 30 19:18:51.532850 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.532069 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0bff1115-e48c-4282-868e-4a572b4c12cf-istiod-ca-cert" pod "0bff1115-e48c-4282-868e-4a572b4c12cf" (UID: "0bff1115-e48c-4282-868e-4a572b4c12cf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:18:51.532850 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.532487 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.532850 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.532512 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.532850 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.532528 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.532850 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.532544 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0bff1115-e48c-4282-868e-4a572b4c12cf-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.532850 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.532558 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.532850 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.532585 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.532850 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.532600 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.532850 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.532615 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.532850 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.532628 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.532850 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.532687 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-workload-certs" pod "1a6b1680-232d-4c50-beee-d3967a408fd4" (UID: "1a6b1680-232d-4c50-beee-d3967a408fd4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.532850 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.532802 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-token" pod "a1d0e8a5-75ba-4e8d-9168-384428b30c4c" (UID: "a1d0e8a5-75ba-4e8d-9168-384428b30c4c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:18:51.533496 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.533126 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-credential-socket" pod "1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" (UID: "1d4f4ad1-2be4-4c2a-8614-1b71cd7365da"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.533496 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.533313 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-data" pod "a1d0e8a5-75ba-4e8d-9168-384428b30c4c" (UID: "a1d0e8a5-75ba-4e8d-9168-384428b30c4c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.533631 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.533560 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-workload-socket" pod "1a6b1680-232d-4c50-beee-d3967a408fd4" (UID: "1a6b1680-232d-4c50-beee-d3967a408fd4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.534188 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.534045 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a6b1680-232d-4c50-beee-d3967a408fd4-istiod-ca-cert" pod "1a6b1680-232d-4c50-beee-d3967a408fd4" (UID: "1a6b1680-232d-4c50-beee-d3967a408fd4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:18:51.534188 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.534177 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-workload-socket" pod "1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" (UID: "1d4f4ad1-2be4-4c2a-8614-1b71cd7365da"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.534710 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.534689 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-workload-certs" pod "0bff1115-e48c-4282-868e-4a572b4c12cf" (UID: "0bff1115-e48c-4282-868e-4a572b4c12cf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.534969 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.534631 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istiod-ca-cert" pod "1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" (UID: "1d4f4ad1-2be4-4c2a-8614-1b71cd7365da"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:18:51.534969 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.534900 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-workload-socket" pod "0bff1115-e48c-4282-868e-4a572b4c12cf" (UID: "0bff1115-e48c-4282-868e-4a572b4c12cf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.535385 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.535217 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-credential-socket" pod "1a6b1680-232d-4c50-beee-d3967a408fd4" (UID: "1a6b1680-232d-4c50-beee-d3967a408fd4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.535810 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.535526 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-data" pod "1a6b1680-232d-4c50-beee-d3967a408fd4" (UID: "1a6b1680-232d-4c50-beee-d3967a408fd4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.537895 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.537831 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-token" pod "1a6b1680-232d-4c50-beee-d3967a408fd4" (UID: "1a6b1680-232d-4c50-beee-d3967a408fd4"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:18:51.538813 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.538740 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-envoy" pod "a1d0e8a5-75ba-4e8d-9168-384428b30c4c" (UID: "a1d0e8a5-75ba-4e8d-9168-384428b30c4c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.538813 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.538761 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-kube-api-access-nrmql" pod "1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" (UID: "1d4f4ad1-2be4-4c2a-8614-1b71cd7365da"). InnerVolumeSpecName "kube-api-access-nrmql". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:18:51.538979 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.538865 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-token" pod "1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" (UID: "1d4f4ad1-2be4-4c2a-8614-1b71cd7365da"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:18:51.539074 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.539053 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-podinfo" pod "1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" (UID: "1d4f4ad1-2be4-4c2a-8614-1b71cd7365da"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:18:51.539786 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.539675 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a6b1680-232d-4c50-beee-d3967a408fd4-kube-api-access-8jw4c" pod "1a6b1680-232d-4c50-beee-d3967a408fd4" (UID: "1a6b1680-232d-4c50-beee-d3967a408fd4"). InnerVolumeSpecName "kube-api-access-8jw4c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:18:51.539786 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.539681 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-podinfo" pod "a1d0e8a5-75ba-4e8d-9168-384428b30c4c" (UID: "a1d0e8a5-75ba-4e8d-9168-384428b30c4c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:18:51.539786 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.539761 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-envoy" pod "0bff1115-e48c-4282-868e-4a572b4c12cf" (UID: "0bff1115-e48c-4282-868e-4a572b4c12cf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.540144 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.540116 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-envoy" pod "1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" (UID: "1d4f4ad1-2be4-4c2a-8614-1b71cd7365da"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.540240 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.540165 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-envoy" pod "1a6b1680-232d-4c50-beee-d3967a408fd4" (UID: "1a6b1680-232d-4c50-beee-d3967a408fd4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:18:51.540240 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.540183 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-kube-api-access-jbpx9" pod "a1d0e8a5-75ba-4e8d-9168-384428b30c4c" (UID: "a1d0e8a5-75ba-4e8d-9168-384428b30c4c"). InnerVolumeSpecName "kube-api-access-jbpx9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:18:51.540373 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.540247 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-podinfo" pod "0bff1115-e48c-4282-868e-4a572b4c12cf" (UID: "0bff1115-e48c-4282-868e-4a572b4c12cf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:18:51.540434 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.540402 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-podinfo" pod "1a6b1680-232d-4c50-beee-d3967a408fd4" (UID: "1a6b1680-232d-4c50-beee-d3967a408fd4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:18:51.540828 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.540811 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-token" pod "0bff1115-e48c-4282-868e-4a572b4c12cf" (UID: "0bff1115-e48c-4282-868e-4a572b4c12cf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:18:51.541942 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.541923 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bff1115-e48c-4282-868e-4a572b4c12cf-kube-api-access-p6xdq" pod "0bff1115-e48c-4282-868e-4a572b4c12cf" (UID: "0bff1115-e48c-4282-868e-4a572b4c12cf"). InnerVolumeSpecName "kube-api-access-p6xdq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:18:51.633280 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633214 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633280 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633239 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633280 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633248 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633280 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633256 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633280 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633265 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633280 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633274 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633280 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633282 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633280 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633291 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nrmql\" (UniqueName: \"kubernetes.io/projected/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-kube-api-access-nrmql\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633300 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633309 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633318 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jbpx9\" (UniqueName: \"kubernetes.io/projected/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-kube-api-access-jbpx9\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633327 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633335 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8jw4c\" (UniqueName: \"kubernetes.io/projected/1a6b1680-232d-4c50-beee-d3967a408fd4-kube-api-access-8jw4c\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633345 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633355 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633363 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0bff1115-e48c-4282-868e-4a572b4c12cf-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633371 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633379 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633388 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a6b1680-232d-4c50-beee-d3967a408fd4-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633396 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633404 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633412 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a6b1680-232d-4c50-beee-d3967a408fd4-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633420 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0bff1115-e48c-4282-868e-4a572b4c12cf-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633429 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a6b1680-232d-4c50-beee-d3967a408fd4-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633437 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6xdq\" (UniqueName: \"kubernetes.io/projected/0bff1115-e48c-4282-868e-4a572b4c12cf-kube-api-access-p6xdq\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633445 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.633840 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.633452 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a1d0e8a5-75ba-4e8d-9168-384428b30c4c-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:51.650150 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:51.650112 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e4b4e21b-a660-4b77-869c-64ec2a9996be" path="/var/lib/kubelet/pods/e4b4e21b-a660-4b77-869c-64ec2a9996be/volumes" Oct 30 19:18:52.178553 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.178516 5894 generic.go:356] "Generic (PLEG): container finished" podID="1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" containerID="551910f8f6104ea23e3ee182d2863d1fd37b505e613e49aa1f8216fc8da79a07" exitCode=0 Oct 30 19:18:52.178751 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.178564 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" Oct 30 19:18:52.178751 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.178603 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" event={"ID":"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da","Type":"ContainerDied","Data":"551910f8f6104ea23e3ee182d2863d1fd37b505e613e49aa1f8216fc8da79a07"} Oct 30 19:18:52.178751 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.178662 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5" event={"ID":"1d4f4ad1-2be4-4c2a-8614-1b71cd7365da","Type":"ContainerDied","Data":"651a519cf6b5f13afb4c6e532350242fd87ce09c50bf027165c7b9c9af404ada"} Oct 30 19:18:52.178751 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.178685 5894 scope.go:122] "RemoveContainer" containerID="551910f8f6104ea23e3ee182d2863d1fd37b505e613e49aa1f8216fc8da79a07" Oct 30 19:18:52.180290 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.180267 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" Oct 30 19:18:52.180423 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.180288 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d" event={"ID":"1a6b1680-232d-4c50-beee-d3967a408fd4","Type":"ContainerDied","Data":"be78832875ca73a21d632da72b6d243b1c38d4918b0dd8336d3f4ff10662bda9"} Oct 30 19:18:52.182069 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.182049 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" Oct 30 19:18:52.182779 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.182564 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d" event={"ID":"a1d0e8a5-75ba-4e8d-9168-384428b30c4c","Type":"ContainerDied","Data":"cbe49f966692926c5667e6c778905b2ed4da840d61bce6f25002cf3e40086a2b"} Oct 30 19:18:52.184064 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.184041 5894 generic.go:356] "Generic (PLEG): container finished" podID="0bff1115-e48c-4282-868e-4a572b4c12cf" containerID="d9963cdea518f4510e4e475f270bbab446724f750cc6c9e65e9ec19f58b81f70" exitCode=0 Oct 30 19:18:52.184189 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.184077 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" event={"ID":"0bff1115-e48c-4282-868e-4a572b4c12cf","Type":"ContainerDied","Data":"d9963cdea518f4510e4e475f270bbab446724f750cc6c9e65e9ec19f58b81f70"} Oct 30 19:18:52.184189 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.184087 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" Oct 30 19:18:52.184189 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.184106 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27" event={"ID":"0bff1115-e48c-4282-868e-4a572b4c12cf","Type":"ContainerDied","Data":"11b1066b7c7dc66b4ecdbc62a9a98c6c0c7a3b70103bba72559af537b59aeb94"} Oct 30 19:18:52.194644 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.194623 5894 scope.go:122] "RemoveContainer" containerID="551910f8f6104ea23e3ee182d2863d1fd37b505e613e49aa1f8216fc8da79a07" Oct 30 19:18:52.196511 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:18:52.196489 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"551910f8f6104ea23e3ee182d2863d1fd37b505e613e49aa1f8216fc8da79a07\": container with ID starting with 551910f8f6104ea23e3ee182d2863d1fd37b505e613e49aa1f8216fc8da79a07 not found: ID does not exist" containerID="551910f8f6104ea23e3ee182d2863d1fd37b505e613e49aa1f8216fc8da79a07" Oct 30 19:18:52.196511 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.196519 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"551910f8f6104ea23e3ee182d2863d1fd37b505e613e49aa1f8216fc8da79a07"} err="failed to get container status \"551910f8f6104ea23e3ee182d2863d1fd37b505e613e49aa1f8216fc8da79a07\": rpc error: code = NotFound desc = could not find container \"551910f8f6104ea23e3ee182d2863d1fd37b505e613e49aa1f8216fc8da79a07\": container with ID starting with 551910f8f6104ea23e3ee182d2863d1fd37b505e613e49aa1f8216fc8da79a07 not found: ID does not exist" Oct 30 19:18:52.196511 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.196541 5894 scope.go:122] "RemoveContainer" containerID="03975e70e8168300f82971dfc7ed26bd583c25fd8d0a5b872ba6331498f90188" Oct 30 19:18:52.207136 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.207120 5894 scope.go:122] "RemoveContainer" containerID="5df39f9c3205bd39046421c940f23a02837a64781b9c56d8efc7e827617eacbf" Oct 30 19:18:52.216563 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.216549 5894 scope.go:122] "RemoveContainer" containerID="d9963cdea518f4510e4e475f270bbab446724f750cc6c9e65e9ec19f58b81f70" Oct 30 19:18:52.225530 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.225514 5894 scope.go:122] "RemoveContainer" containerID="d9963cdea518f4510e4e475f270bbab446724f750cc6c9e65e9ec19f58b81f70" Oct 30 19:18:52.225790 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:18:52.225770 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9963cdea518f4510e4e475f270bbab446724f750cc6c9e65e9ec19f58b81f70\": container with ID starting with d9963cdea518f4510e4e475f270bbab446724f750cc6c9e65e9ec19f58b81f70 not found: ID does not exist" containerID="d9963cdea518f4510e4e475f270bbab446724f750cc6c9e65e9ec19f58b81f70" Oct 30 19:18:52.225844 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.225798 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9963cdea518f4510e4e475f270bbab446724f750cc6c9e65e9ec19f58b81f70"} err="failed to get container status \"d9963cdea518f4510e4e475f270bbab446724f750cc6c9e65e9ec19f58b81f70\": rpc error: code = NotFound desc = could not find container \"d9963cdea518f4510e4e475f270bbab446724f750cc6c9e65e9ec19f58b81f70\": container with ID starting with d9963cdea518f4510e4e475f270bbab446724f750cc6c9e65e9ec19f58b81f70 not found: ID does not exist" Oct 30 19:18:52.342618 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.342505 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d"] Oct 30 19:18:52.392090 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.392051 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-tln5d"] Oct 30 19:18:52.455303 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.455273 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d"] Oct 30 19:18:52.461310 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.461280 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-x6n7d"] Oct 30 19:18:52.580976 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.580946 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27"] Oct 30 19:18:52.599648 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.599561 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-46v27"] Oct 30 19:18:52.708939 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.708904 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5"] Oct 30 19:18:52.714885 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.714858 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-ae6a138d-openshift-default-557d46bb4f-rgjs5"] Oct 30 19:18:52.917601 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:52.917559 5894 scope.go:122] "RemoveContainer" containerID="dc633cfa9302d94206772134950af8f1055edcabf0ee8a9813db39b5cec4f282" Oct 30 19:18:53.264198 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:53.264051 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:53.649640 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:53.649611 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="0bff1115-e48c-4282-868e-4a572b4c12cf" path="/var/lib/kubelet/pods/0bff1115-e48c-4282-868e-4a572b4c12cf/volumes" Oct 30 19:18:53.650094 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:53.650079 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1a6b1680-232d-4c50-beee-d3967a408fd4" path="/var/lib/kubelet/pods/1a6b1680-232d-4c50-beee-d3967a408fd4/volumes" Oct 30 19:18:53.650500 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:53.650487 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" path="/var/lib/kubelet/pods/1d4f4ad1-2be4-4c2a-8614-1b71cd7365da/volumes" Oct 30 19:18:53.651257 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:53.651241 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a1d0e8a5-75ba-4e8d-9168-384428b30c4c" path="/var/lib/kubelet/pods/a1d0e8a5-75ba-4e8d-9168-384428b30c4c/volumes" Oct 30 19:18:57.161374 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:57.161337 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4"] Oct 30 19:18:57.161899 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:57.161604 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" podUID="6fb6a97e-220d-4b2b-aa99-7732009c8157" containerName="maas-api" containerID="cri-o://8d217fbceb28cf7f46f2df3725b22ca20742c282f6ad8b2bf552f2b59f3a2a70" gracePeriod=30 Oct 30 19:18:57.299566 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:57.299528 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:57.480471 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:57.480445 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" Oct 30 19:18:57.504109 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:57.504075 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6fb6a97e-220d-4b2b-aa99-7732009c8157-metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fb6a97e-220d-4b2b-aa99-7732009c8157-metrics-certs\") pod \"6fb6a97e-220d-4b2b-aa99-7732009c8157\" (UID: \"6fb6a97e-220d-4b2b-aa99-7732009c8157\") " Oct 30 19:18:57.504296 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:57.504142 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/6fb6a97e-220d-4b2b-aa99-7732009c8157-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb6a97e-220d-4b2b-aa99-7732009c8157-maas-api-tls\") pod \"6fb6a97e-220d-4b2b-aa99-7732009c8157\" (UID: \"6fb6a97e-220d-4b2b-aa99-7732009c8157\") " Oct 30 19:18:57.504296 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:57.504191 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6fb6a97e-220d-4b2b-aa99-7732009c8157-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb6a97e-220d-4b2b-aa99-7732009c8157-openshift-service-ca\") pod \"6fb6a97e-220d-4b2b-aa99-7732009c8157\" (UID: \"6fb6a97e-220d-4b2b-aa99-7732009c8157\") " Oct 30 19:18:57.504418 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:57.504327 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6fb6a97e-220d-4b2b-aa99-7732009c8157-kube-api-access-vmzdg\" (UniqueName: \"kubernetes.io/projected/6fb6a97e-220d-4b2b-aa99-7732009c8157-kube-api-access-vmzdg\") pod \"6fb6a97e-220d-4b2b-aa99-7732009c8157\" (UID: \"6fb6a97e-220d-4b2b-aa99-7732009c8157\") " Oct 30 19:18:57.504847 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:57.504823 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fb6a97e-220d-4b2b-aa99-7732009c8157-openshift-service-ca" pod "6fb6a97e-220d-4b2b-aa99-7732009c8157" (UID: "6fb6a97e-220d-4b2b-aa99-7732009c8157"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:18:57.506921 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:57.506894 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fb6a97e-220d-4b2b-aa99-7732009c8157-kube-api-access-vmzdg" pod "6fb6a97e-220d-4b2b-aa99-7732009c8157" (UID: "6fb6a97e-220d-4b2b-aa99-7732009c8157"). InnerVolumeSpecName "kube-api-access-vmzdg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:18:57.507058 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:57.507023 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fb6a97e-220d-4b2b-aa99-7732009c8157-metrics-certs" pod "6fb6a97e-220d-4b2b-aa99-7732009c8157" (UID: "6fb6a97e-220d-4b2b-aa99-7732009c8157"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:18:57.507278 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:57.507252 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fb6a97e-220d-4b2b-aa99-7732009c8157-maas-api-tls" pod "6fb6a97e-220d-4b2b-aa99-7732009c8157" (UID: "6fb6a97e-220d-4b2b-aa99-7732009c8157"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:18:57.606466 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:57.606435 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb6a97e-220d-4b2b-aa99-7732009c8157-maas-api-tls\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:57.606466 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:57.606463 5894 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb6a97e-220d-4b2b-aa99-7732009c8157-openshift-service-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:57.606676 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:57.606474 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmzdg\" (UniqueName: \"kubernetes.io/projected/6fb6a97e-220d-4b2b-aa99-7732009c8157-kube-api-access-vmzdg\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:57.606676 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:57.606484 5894 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6fb6a97e-220d-4b2b-aa99-7732009c8157-metrics-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:18:58.218202 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:58.218171 5894 generic.go:356] "Generic (PLEG): container finished" podID="6fb6a97e-220d-4b2b-aa99-7732009c8157" containerID="8d217fbceb28cf7f46f2df3725b22ca20742c282f6ad8b2bf552f2b59f3a2a70" exitCode=0 Oct 30 19:18:58.218698 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:58.218217 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" Oct 30 19:18:58.218698 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:58.218254 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" event={"ID":"6fb6a97e-220d-4b2b-aa99-7732009c8157","Type":"ContainerDied","Data":"8d217fbceb28cf7f46f2df3725b22ca20742c282f6ad8b2bf552f2b59f3a2a70"} Oct 30 19:18:58.218698 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:58.218317 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4" event={"ID":"6fb6a97e-220d-4b2b-aa99-7732009c8157","Type":"ContainerDied","Data":"abbce4322541c0fb89f2c166e3670c9fb6296b413d2af93500e4ff8dc246797d"} Oct 30 19:18:58.218698 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:58.218343 5894 scope.go:122] "RemoveContainer" containerID="8d217fbceb28cf7f46f2df3725b22ca20742c282f6ad8b2bf552f2b59f3a2a70" Oct 30 19:18:58.227681 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:58.227662 5894 scope.go:122] "RemoveContainer" containerID="8d217fbceb28cf7f46f2df3725b22ca20742c282f6ad8b2bf552f2b59f3a2a70" Oct 30 19:18:58.227946 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:18:58.227922 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d217fbceb28cf7f46f2df3725b22ca20742c282f6ad8b2bf552f2b59f3a2a70\": container with ID starting with 8d217fbceb28cf7f46f2df3725b22ca20742c282f6ad8b2bf552f2b59f3a2a70 not found: ID does not exist" containerID="8d217fbceb28cf7f46f2df3725b22ca20742c282f6ad8b2bf552f2b59f3a2a70" Oct 30 19:18:58.227995 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:58.227954 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d217fbceb28cf7f46f2df3725b22ca20742c282f6ad8b2bf552f2b59f3a2a70"} err="failed to get container status \"8d217fbceb28cf7f46f2df3725b22ca20742c282f6ad8b2bf552f2b59f3a2a70\": rpc error: code = NotFound desc = could not find container \"8d217fbceb28cf7f46f2df3725b22ca20742c282f6ad8b2bf552f2b59f3a2a70\": container with ID starting with 8d217fbceb28cf7f46f2df3725b22ca20742c282f6ad8b2bf552f2b59f3a2a70 not found: ID does not exist" Oct 30 19:18:58.381984 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:58.381943 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4"] Oct 30 19:18:58.431859 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:58.431824 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w1-975cf5-84b97547f6-7qrr4"] Oct 30 19:18:58.818010 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:58.817977 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:59.024671 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:59.024635 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:59.207777 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:59.207746 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:18:59.650030 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:59.649994 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6fb6a97e-220d-4b2b-aa99-7732009c8157" path="/var/lib/kubelet/pods/6fb6a97e-220d-4b2b-aa99-7732009c8157/volumes" Oct 30 19:18:59.785977 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:18:59.785946 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:03.814927 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:03.814895 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:04.072130 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:04.072038 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:05.741753 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:05.741714 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:05.917790 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:05.917755 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:06.048135 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:06.048034 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm"] Oct 30 19:19:06.048419 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:06.048388 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" podUID="6364da0d-986e-4551-ae12-b4f3f6963799" containerName="istio-proxy" containerID="cri-o://3694dbd41ad629bf0d2638e0870bd49a522215392fd4a54a9e1aee6257ed781d" gracePeriod=30 Oct 30 19:19:06.119360 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:06.119288 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:08.733429 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:08.733394 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn"] Oct 30 19:19:08.878094 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:08.878066 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:09.188087 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:09.188055 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:11.300728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.300695 5894 generic.go:356] "Generic (PLEG): container finished" podID="6364da0d-986e-4551-ae12-b4f3f6963799" containerID="3694dbd41ad629bf0d2638e0870bd49a522215392fd4a54a9e1aee6257ed781d" exitCode=0 Oct 30 19:19:11.301073 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.300774 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" event={"ID":"6364da0d-986e-4551-ae12-b4f3f6963799","Type":"ContainerDied","Data":"3694dbd41ad629bf0d2638e0870bd49a522215392fd4a54a9e1aee6257ed781d"} Oct 30 19:19:11.301073 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.300866 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" event={"ID":"6364da0d-986e-4551-ae12-b4f3f6963799","Type":"ContainerDied","Data":"24f401004935d5a65c764b69bfbd3ef5b16df7af83bb460122e3b76846af5d3c"} Oct 30 19:19:11.301073 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.300886 5894 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="24f401004935d5a65c764b69bfbd3ef5b16df7af83bb460122e3b76846af5d3c" Oct 30 19:19:11.306304 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.306284 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:19:11.387502 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.387438 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6364da0d-986e-4551-ae12-b4f3f6963799-istio-token\" (UniqueName: \"kubernetes.io/projected/6364da0d-986e-4551-ae12-b4f3f6963799-istio-token\") pod \"6364da0d-986e-4551-ae12-b4f3f6963799\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " Oct 30 19:19:11.387630 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.387504 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-workload-certs\") pod \"6364da0d-986e-4551-ae12-b4f3f6963799\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " Oct 30 19:19:11.387630 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.387557 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/6364da0d-986e-4551-ae12-b4f3f6963799-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6364da0d-986e-4551-ae12-b4f3f6963799-istiod-ca-cert\") pod \"6364da0d-986e-4551-ae12-b4f3f6963799\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " Oct 30 19:19:11.387630 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.387613 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/6364da0d-986e-4551-ae12-b4f3f6963799-kube-api-access-hm2bg\" (UniqueName: \"kubernetes.io/projected/6364da0d-986e-4551-ae12-b4f3f6963799-kube-api-access-hm2bg\") pod \"6364da0d-986e-4551-ae12-b4f3f6963799\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " Oct 30 19:19:11.387869 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.387639 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-workload-socket\") pod \"6364da0d-986e-4551-ae12-b4f3f6963799\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " Oct 30 19:19:11.387869 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.387668 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/6364da0d-986e-4551-ae12-b4f3f6963799-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6364da0d-986e-4551-ae12-b4f3f6963799-istio-podinfo\") pod \"6364da0d-986e-4551-ae12-b4f3f6963799\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " Oct 30 19:19:11.387869 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.387767 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-credential-socket\") pod \"6364da0d-986e-4551-ae12-b4f3f6963799\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " Oct 30 19:19:11.387869 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.387808 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-istio-envoy\") pod \"6364da0d-986e-4551-ae12-b4f3f6963799\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " Oct 30 19:19:11.387869 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.387824 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-workload-certs" pod "6364da0d-986e-4551-ae12-b4f3f6963799" (UID: "6364da0d-986e-4551-ae12-b4f3f6963799"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:19:11.387869 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.387861 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-istio-data\") pod \"6364da0d-986e-4551-ae12-b4f3f6963799\" (UID: \"6364da0d-986e-4551-ae12-b4f3f6963799\") " Oct 30 19:19:11.388179 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.387992 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-workload-socket" pod "6364da0d-986e-4551-ae12-b4f3f6963799" (UID: "6364da0d-986e-4551-ae12-b4f3f6963799"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:19:11.388179 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.388089 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6364da0d-986e-4551-ae12-b4f3f6963799-istiod-ca-cert" pod "6364da0d-986e-4551-ae12-b4f3f6963799" (UID: "6364da0d-986e-4551-ae12-b4f3f6963799"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:19:11.388179 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.388104 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-credential-socket" pod "6364da0d-986e-4551-ae12-b4f3f6963799" (UID: "6364da0d-986e-4551-ae12-b4f3f6963799"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:19:11.388312 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.388231 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-istio-data" pod "6364da0d-986e-4551-ae12-b4f3f6963799" (UID: "6364da0d-986e-4551-ae12-b4f3f6963799"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:19:11.388376 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.388329 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6364da0d-986e-4551-ae12-b4f3f6963799-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:19:11.388376 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.388346 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:19:11.388376 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.388355 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:19:11.388376 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.388365 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:19:11.388376 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.388376 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:19:11.390054 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.390027 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-istio-envoy" pod "6364da0d-986e-4551-ae12-b4f3f6963799" (UID: "6364da0d-986e-4551-ae12-b4f3f6963799"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:19:11.390054 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.390032 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6364da0d-986e-4551-ae12-b4f3f6963799-kube-api-access-hm2bg" pod "6364da0d-986e-4551-ae12-b4f3f6963799" (UID: "6364da0d-986e-4551-ae12-b4f3f6963799"). InnerVolumeSpecName "kube-api-access-hm2bg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:19:11.390211 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.390178 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6364da0d-986e-4551-ae12-b4f3f6963799-istio-token" pod "6364da0d-986e-4551-ae12-b4f3f6963799" (UID: "6364da0d-986e-4551-ae12-b4f3f6963799"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:19:11.390261 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.390236 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6364da0d-986e-4551-ae12-b4f3f6963799-istio-podinfo" pod "6364da0d-986e-4551-ae12-b4f3f6963799" (UID: "6364da0d-986e-4551-ae12-b4f3f6963799"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:19:11.489592 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.489550 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6364da0d-986e-4551-ae12-b4f3f6963799-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:19:11.489743 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.489608 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6364da0d-986e-4551-ae12-b4f3f6963799-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:19:11.489743 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.489623 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hm2bg\" (UniqueName: \"kubernetes.io/projected/6364da0d-986e-4551-ae12-b4f3f6963799-kube-api-access-hm2bg\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:19:11.489743 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:11.489639 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6364da0d-986e-4551-ae12-b4f3f6963799-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:19:12.306238 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:12.306205 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm" Oct 30 19:19:12.519852 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:12.519820 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm"] Oct 30 19:19:12.522204 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:12.522177 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-975cf5-openshift-default-5fcdb49bfc-qp7jm"] Oct 30 19:19:13.438822 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:13.438778 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:13.650520 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:13.650487 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="6364da0d-986e-4551-ae12-b4f3f6963799" path="/var/lib/kubelet/pods/6364da0d-986e-4551-ae12-b4f3f6963799/volumes" Oct 30 19:19:15.070826 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:15.070794 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:15.346376 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:15.346303 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:15.928205 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:15.928168 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:16.164302 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:16.164265 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:16.406920 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:16.406886 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:16.588110 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:16.588073 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:17.206097 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:17.206066 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:19.661983 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:19.661948 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:20.117933 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:19:20.117851 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:19:21.508728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:21.508697 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:21.965053 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:21.965013 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:22.259275 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.259174 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.461480 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg"] Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462285 5894 cpu_manager.go:414] "containerMap: removing container" podUID="2a4529e2-c317-4d6d-916f-b77baf48009a" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462307 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a4529e2-c317-4d6d-916f-b77baf48009a" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462326 5894 cpu_manager.go:414] "containerMap: removing container" podUID="48092d6e-157f-4df3-828b-497746965c69" containerName="main" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462334 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="48092d6e-157f-4df3-828b-497746965c69" containerName="main" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462347 5894 cpu_manager.go:414] "containerMap: removing container" podUID="e4b4e21b-a660-4b77-869c-64ec2a9996be" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462354 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e4b4e21b-a660-4b77-869c-64ec2a9996be" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462365 5894 cpu_manager.go:414] "containerMap: removing container" podUID="0bff1115-e48c-4282-868e-4a572b4c12cf" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462372 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="0bff1115-e48c-4282-868e-4a572b4c12cf" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462385 5894 cpu_manager.go:414] "containerMap: removing container" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462392 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462401 5894 cpu_manager.go:414] "containerMap: removing container" podUID="03ef4756-9a48-4f73-9229-6e7cc936e052" containerName="maas-api" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462408 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="03ef4756-9a48-4f73-9229-6e7cc936e052" containerName="maas-api" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462426 5894 cpu_manager.go:414] "containerMap: removing container" podUID="787838ab-7e4d-423b-be79-b34816b39224" containerName="maas-api" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462442 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="787838ab-7e4d-423b-be79-b34816b39224" containerName="maas-api" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462454 5894 cpu_manager.go:414] "containerMap: removing container" podUID="1a6b1680-232d-4c50-beee-d3967a408fd4" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462462 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1a6b1680-232d-4c50-beee-d3967a408fd4" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462483 5894 cpu_manager.go:414] "containerMap: removing container" podUID="2a4529e2-c317-4d6d-916f-b77baf48009a" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462490 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="2a4529e2-c317-4d6d-916f-b77baf48009a" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462507 5894 cpu_manager.go:414] "containerMap: removing container" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462513 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462523 5894 cpu_manager.go:414] "containerMap: removing container" podUID="9d683e67-87f6-4d4b-a208-0ebea816c5a7" containerName="payload-pre-processing" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462531 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9d683e67-87f6-4d4b-a208-0ebea816c5a7" containerName="payload-pre-processing" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462547 5894 cpu_manager.go:414] "containerMap: removing container" podUID="3df940b4-b684-4a85-ae0c-2fe4ffe51849" containerName="payload-pre-processing" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462555 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3df940b4-b684-4a85-ae0c-2fe4ffe51849" containerName="payload-pre-processing" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462568 5894 cpu_manager.go:414] "containerMap: removing container" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462592 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462610 5894 cpu_manager.go:414] "containerMap: removing container" podUID="8a237b2f-38a2-47b6-a0a4-3d89b557b419" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462618 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a237b2f-38a2-47b6-a0a4-3d89b557b419" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462629 5894 cpu_manager.go:414] "containerMap: removing container" podUID="1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462638 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462658 5894 cpu_manager.go:414] "containerMap: removing container" podUID="14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462665 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462677 5894 cpu_manager.go:414] "containerMap: removing container" podUID="bb0794ea-65d0-4050-81eb-2b961d482d33" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462684 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bb0794ea-65d0-4050-81eb-2b961d482d33" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462693 5894 cpu_manager.go:414] "containerMap: removing container" podUID="df3270ce-707c-45f4-a0d9-3dfd137fbdfc" containerName="maas-api" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462700 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="df3270ce-707c-45f4-a0d9-3dfd137fbdfc" containerName="maas-api" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462712 5894 cpu_manager.go:414] "containerMap: removing container" podUID="6fb6a97e-220d-4b2b-aa99-7732009c8157" containerName="maas-api" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462719 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6fb6a97e-220d-4b2b-aa99-7732009c8157" containerName="maas-api" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462756 5894 cpu_manager.go:414] "containerMap: removing container" podUID="c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462765 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462780 5894 cpu_manager.go:414] "containerMap: removing container" podUID="8a237b2f-38a2-47b6-a0a4-3d89b557b419" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462788 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8a237b2f-38a2-47b6-a0a4-3d89b557b419" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462798 5894 cpu_manager.go:414] "containerMap: removing container" podUID="6364da0d-986e-4551-ae12-b4f3f6963799" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462805 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="6364da0d-986e-4551-ae12-b4f3f6963799" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462815 5894 cpu_manager.go:414] "containerMap: removing container" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462823 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462833 5894 cpu_manager.go:414] "containerMap: removing container" podUID="a1d0e8a5-75ba-4e8d-9168-384428b30c4c" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462841 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a1d0e8a5-75ba-4e8d-9168-384428b30c4c" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462859 5894 cpu_manager.go:414] "containerMap: removing container" podUID="ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" containerName="main" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462866 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" containerName="main" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.462985 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="14cda751-f1b9-4b48-9fe2-bcd0a2909d0c" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463000 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="bb0794ea-65d0-4050-81eb-2b961d482d33" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463011 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="df3270ce-707c-45f4-a0d9-3dfd137fbdfc" containerName="maas-api" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463021 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="6fb6a97e-220d-4b2b-aa99-7732009c8157" containerName="maas-api" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463032 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="c7acd06a-8759-4b0a-9e4e-7874e9fd38e5" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463042 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a237b2f-38a2-47b6-a0a4-3d89b557b419" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463050 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="6364da0d-986e-4551-ae12-b4f3f6963799" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463059 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="ee442ed1-4b79-47ab-b0d9-00cc2b0d717a" containerName="main" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463069 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="a1d0e8a5-75ba-4e8d-9168-384428b30c4c" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463079 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="ce4ba09e-cbff-413e-b8cd-17e8de0ce8a2" containerName="main" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463087 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a4529e2-c317-4d6d-916f-b77baf48009a" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463096 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="48092d6e-157f-4df3-828b-497746965c69" containerName="main" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463106 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="e4b4e21b-a660-4b77-869c-64ec2a9996be" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463115 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="0bff1115-e48c-4282-868e-4a572b4c12cf" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463123 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="4049821e-28c6-4718-9ed9-01101012c150" containerName="main" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463131 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="03ef4756-9a48-4f73-9229-6e7cc936e052" containerName="maas-api" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463138 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="787838ab-7e4d-423b-be79-b34816b39224" containerName="maas-api" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463147 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="1a6b1680-232d-4c50-beee-d3967a408fd4" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463155 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="2a4529e2-c317-4d6d-916f-b77baf48009a" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463166 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="67c4ee9e-c041-4f48-936b-afb5a69dbd1f" containerName="main" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463174 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="9d683e67-87f6-4d4b-a208-0ebea816c5a7" containerName="payload-pre-processing" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463183 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="3df940b4-b684-4a85-ae0c-2fe4ffe51849" containerName="payload-pre-processing" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463193 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="b42c4711-cc79-4024-b2b9-c6294004c373" containerName="main" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463201 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="8a237b2f-38a2-47b6-a0a4-3d89b557b419" containerName="istio-proxy" Oct 30 19:19:22.463613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.463211 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="1d4f4ad1-2be4-4c2a-8614-1b71cd7365da" containerName="istio-proxy" Oct 30 19:19:22.469704 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.469650 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:22.489852 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.489812 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.510690 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.510623 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl"] Oct 30 19:19:22.514404 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.514381 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s"] Oct 30 19:19:22.514544 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.514526 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.518410 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.518389 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.525949 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.525925 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg"] Oct 30 19:19:22.557923 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.557864 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-qwc8z\"" Oct 30 19:19:22.583270 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.583212 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl"] Oct 30 19:19:22.584553 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.584530 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s"] Oct 30 19:19:22.617204 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.617172 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75f81dfd-86e7-4532-a20c-42423f908903-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.617330 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.617207 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/010f73a2-9a20-49a6-8d88-079a27465e78-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.617330 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.617241 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/010f73a2-9a20-49a6-8d88-079a27465e78-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.617419 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.617390 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/010f73a2-9a20-49a6-8d88-079a27465e78-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.617506 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.617480 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fec8f46-c372-49cc-8c0a-9cb885f465be-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.617599 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.617564 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7fr4l\" (UniqueName: \"kubernetes.io/projected/3fec8f46-c372-49cc-8c0a-9cb885f465be-kube-api-access-7fr4l\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.617655 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.617640 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/010f73a2-9a20-49a6-8d88-079a27465e78-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.617694 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.617671 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fec8f46-c372-49cc-8c0a-9cb885f465be-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.617751 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.617730 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75f81dfd-86e7-4532-a20c-42423f908903-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.617791 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.617772 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75f81dfd-86e7-4532-a20c-42423f908903-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.617861 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.617839 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75f81dfd-86e7-4532-a20c-42423f908903-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.617914 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.617898 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/010f73a2-9a20-49a6-8d88-079a27465e78-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.617950 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.617924 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75f81dfd-86e7-4532-a20c-42423f908903-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.617950 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.617941 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75f81dfd-86e7-4532-a20c-42423f908903-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.618052 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.617963 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fec8f46-c372-49cc-8c0a-9cb885f465be-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.618105 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.618089 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-685t9\" (UniqueName: \"kubernetes.io/projected/010f73a2-9a20-49a6-8d88-079a27465e78-kube-api-access-685t9\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.618160 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.618116 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/010f73a2-9a20-49a6-8d88-079a27465e78-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.618160 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.618134 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75f81dfd-86e7-4532-a20c-42423f908903-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.618264 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.618174 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fec8f46-c372-49cc-8c0a-9cb885f465be-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.618300 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.618283 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fec8f46-c372-49cc-8c0a-9cb885f465be-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.618334 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.618316 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fec8f46-c372-49cc-8c0a-9cb885f465be-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.618444 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.618415 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/010f73a2-9a20-49a6-8d88-079a27465e78-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.618512 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.618472 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fec8f46-c372-49cc-8c0a-9cb885f465be-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.618567 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.618529 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/010f73a2-9a20-49a6-8d88-079a27465e78-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.618643 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.618624 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75f81dfd-86e7-4532-a20c-42423f908903-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.618679 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.618657 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fec8f46-c372-49cc-8c0a-9cb885f465be-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.618722 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.618692 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zppjh\" (UniqueName: \"kubernetes.io/projected/75f81dfd-86e7-4532-a20c-42423f908903-kube-api-access-zppjh\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.719566 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.719533 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zppjh\" (UniqueName: \"kubernetes.io/projected/75f81dfd-86e7-4532-a20c-42423f908903-kube-api-access-zppjh\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.719760 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.719626 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75f81dfd-86e7-4532-a20c-42423f908903-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.719760 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.719647 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/010f73a2-9a20-49a6-8d88-079a27465e78-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.719760 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.719670 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/010f73a2-9a20-49a6-8d88-079a27465e78-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.719760 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.719698 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/010f73a2-9a20-49a6-8d88-079a27465e78-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.719760 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.719731 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fec8f46-c372-49cc-8c0a-9cb885f465be-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.719762 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7fr4l\" (UniqueName: \"kubernetes.io/projected/3fec8f46-c372-49cc-8c0a-9cb885f465be-kube-api-access-7fr4l\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.719806 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/010f73a2-9a20-49a6-8d88-079a27465e78-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.719926 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fec8f46-c372-49cc-8c0a-9cb885f465be-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.719980 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75f81dfd-86e7-4532-a20c-42423f908903-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720146 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/010f73a2-9a20-49a6-8d88-079a27465e78-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720192 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75f81dfd-86e7-4532-a20c-42423f908903-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720220 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75f81dfd-86e7-4532-a20c-42423f908903-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720259 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/010f73a2-9a20-49a6-8d88-079a27465e78-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720284 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75f81dfd-86e7-4532-a20c-42423f908903-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720311 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75f81dfd-86e7-4532-a20c-42423f908903-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720296 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/010f73a2-9a20-49a6-8d88-079a27465e78-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720343 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fec8f46-c372-49cc-8c0a-9cb885f465be-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720393 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3fec8f46-c372-49cc-8c0a-9cb885f465be-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720191 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/010f73a2-9a20-49a6-8d88-079a27465e78-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720398 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-685t9\" (UniqueName: \"kubernetes.io/projected/010f73a2-9a20-49a6-8d88-079a27465e78-kube-api-access-685t9\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720396 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/75f81dfd-86e7-4532-a20c-42423f908903-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720525 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/010f73a2-9a20-49a6-8d88-079a27465e78-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720569 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75f81dfd-86e7-4532-a20c-42423f908903-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720619 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fec8f46-c372-49cc-8c0a-9cb885f465be-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720657 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fec8f46-c372-49cc-8c0a-9cb885f465be-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720684 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fec8f46-c372-49cc-8c0a-9cb885f465be-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720734 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/010f73a2-9a20-49a6-8d88-079a27465e78-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.720728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720737 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fec8f46-c372-49cc-8c0a-9cb885f465be-credential-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.721974 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720771 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fec8f46-c372-49cc-8c0a-9cb885f465be-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.721974 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.720952 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/75f81dfd-86e7-4532-a20c-42423f908903-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.721974 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.721138 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/75f81dfd-86e7-4532-a20c-42423f908903-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.721974 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.721227 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/75f81dfd-86e7-4532-a20c-42423f908903-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.721974 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.721339 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3fec8f46-c372-49cc-8c0a-9cb885f465be-workload-socket\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.721974 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.721403 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/010f73a2-9a20-49a6-8d88-079a27465e78-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.721974 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.721448 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75f81dfd-86e7-4532-a20c-42423f908903-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.721974 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.721473 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fec8f46-c372-49cc-8c0a-9cb885f465be-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.721974 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.721712 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/010f73a2-9a20-49a6-8d88-079a27465e78-workload-certs\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.721974 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.721808 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3fec8f46-c372-49cc-8c0a-9cb885f465be-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.721974 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.721812 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3fec8f46-c372-49cc-8c0a-9cb885f465be-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.721974 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.721961 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/75f81dfd-86e7-4532-a20c-42423f908903-istio-data\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.722720 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.722698 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/010f73a2-9a20-49a6-8d88-079a27465e78-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.722794 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.722773 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/75f81dfd-86e7-4532-a20c-42423f908903-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.723201 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.723180 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/75f81dfd-86e7-4532-a20c-42423f908903-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.723295 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.723191 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3fec8f46-c372-49cc-8c0a-9cb885f465be-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.723461 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.723440 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3fec8f46-c372-49cc-8c0a-9cb885f465be-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.724642 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.724622 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/010f73a2-9a20-49a6-8d88-079a27465e78-istio-podinfo\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.724879 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.724859 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/010f73a2-9a20-49a6-8d88-079a27465e78-istio-envoy\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.776943 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.776867 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zppjh\" (UniqueName: \"kubernetes.io/projected/75f81dfd-86e7-4532-a20c-42423f908903-kube-api-access-zppjh\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.777097 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.776949 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7fr4l\" (UniqueName: \"kubernetes.io/projected/3fec8f46-c372-49cc-8c0a-9cb885f465be-kube-api-access-7fr4l\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.777359 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.777338 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3fec8f46-c372-49cc-8c0a-9cb885f465be-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-nvv4s\" (UID: \"3fec8f46-c372-49cc-8c0a-9cb885f465be\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:22.777430 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.777369 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/010f73a2-9a20-49a6-8d88-079a27465e78-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.777467 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.777424 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/75f81dfd-86e7-4532-a20c-42423f908903-istio-token\") pod \"maas-default-gateway-openshift-default-6db586dd6c-7z7fl\" (UID: \"75f81dfd-86e7-4532-a20c-42423f908903\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.777615 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.777567 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-685t9\" (UniqueName: \"kubernetes.io/projected/010f73a2-9a20-49a6-8d88-079a27465e78-kube-api-access-685t9\") pod \"maas-default-gateway-openshift-default-6db586dd6c-w22dg\" (UID: \"010f73a2-9a20-49a6-8d88-079a27465e78\") " pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.814096 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.814073 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:22.830470 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.830448 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:22.836620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:22.836567 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:23.231977 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.231952 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg"] Oct 30 19:19:23.233676 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:19:23.233650 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod010f73a2_9a20_49a6_8d88_079a27465e78.slice/crio-32845f6530f507a14ef2f7f009cf686ad9d3eb56ff2426db43c4081200455387 WatchSource:0}: Error finding container 32845f6530f507a14ef2f7f009cf686ad9d3eb56ff2426db43c4081200455387: Status 404 returned error can't find the container with id 32845f6530f507a14ef2f7f009cf686ad9d3eb56ff2426db43c4081200455387 Oct 30 19:19:23.235936 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.235906 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:19:23.236017 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.235972 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:19:23.236017 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.236002 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:19:23.245062 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.245036 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl"] Oct 30 19:19:23.249146 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:19:23.249122 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75f81dfd_86e7_4532_a20c_42423f908903.slice/crio-a0ffbb1f8ff1521f80c6420a4610a2bfbfa9ab2b180370ea9497fc8e8b5613cc WatchSource:0}: Error finding container a0ffbb1f8ff1521f80c6420a4610a2bfbfa9ab2b180370ea9497fc8e8b5613cc: Status 404 returned error can't find the container with id a0ffbb1f8ff1521f80c6420a4610a2bfbfa9ab2b180370ea9497fc8e8b5613cc Oct 30 19:19:23.255320 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.255288 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:19:23.255370 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.255350 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:19:23.255407 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.255376 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:19:23.275829 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.275803 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s"] Oct 30 19:19:23.280210 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:19:23.280185 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fec8f46_c372_49cc_8c0a_9cb885f465be.slice/crio-f4c85dee8f441cd8cc850058242614412cd9017ffc5885ced4025af7da52e2ca WatchSource:0}: Error finding container f4c85dee8f441cd8cc850058242614412cd9017ffc5885ced4025af7da52e2ca: Status 404 returned error can't find the container with id f4c85dee8f441cd8cc850058242614412cd9017ffc5885ced4025af7da52e2ca Oct 30 19:19:23.282637 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.282607 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:19:23.282709 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.282667 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:19:23.282709 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.282700 5894 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236068Ki","pods":"250"} Oct 30 19:19:23.374630 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.374590 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" event={"ID":"3fec8f46-c372-49cc-8c0a-9cb885f465be","Type":"ContainerStarted","Data":"855d787e19404c02dee0aeb2ae762560558b9fc6c3878e9415d3127be2f318f8"} Oct 30 19:19:23.374788 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.374661 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" event={"ID":"3fec8f46-c372-49cc-8c0a-9cb885f465be","Type":"ContainerStarted","Data":"f4c85dee8f441cd8cc850058242614412cd9017ffc5885ced4025af7da52e2ca"} Oct 30 19:19:23.376901 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.376871 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" event={"ID":"010f73a2-9a20-49a6-8d88-079a27465e78","Type":"ContainerStarted","Data":"cb5dac4caf0887fdcea768035a108f6e58eee9696d03f28e19715afe6cd758ae"} Oct 30 19:19:23.377011 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.376923 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" event={"ID":"010f73a2-9a20-49a6-8d88-079a27465e78","Type":"ContainerStarted","Data":"32845f6530f507a14ef2f7f009cf686ad9d3eb56ff2426db43c4081200455387"} Oct 30 19:19:23.385595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.379357 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" event={"ID":"75f81dfd-86e7-4532-a20c-42423f908903","Type":"ContainerStarted","Data":"5067ace78c09976b8dfe6478b7095021c538b7da7b4eec47f2a6176dc784eb26"} Oct 30 19:19:23.385595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.379487 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" event={"ID":"75f81dfd-86e7-4532-a20c-42423f908903","Type":"ContainerStarted","Data":"a0ffbb1f8ff1521f80c6420a4610a2bfbfa9ab2b180370ea9497fc8e8b5613cc"} Oct 30 19:19:23.494821 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.494362 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" podStartSLOduration=1.4943445340000001 podStartE2EDuration="1.494344534s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:23.493950937 +0000 UTC m=+2433.984078784" watchObservedRunningTime="2026-10-30 19:19:23.494344534 +0000 UTC m=+2433.984472354" Oct 30 19:19:23.507688 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.507646 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:23.757016 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.756925 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" containerID="cri-o://22872da387db898771bf377c0134d75a87b8744dc5c42a8d8479c4eacdc5333a" gracePeriod=45 Oct 30 19:19:23.764992 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.764932 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" podStartSLOduration=1.764911042 podStartE2EDuration="1.764911042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:23.614651094 +0000 UTC m=+2434.104778915" watchObservedRunningTime="2026-10-30 19:19:23.764911042 +0000 UTC m=+2434.255038862" Oct 30 19:19:23.765854 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.765782 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" podStartSLOduration=1.765774323 podStartE2EDuration="1.765774323s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:19:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:19:23.759507932 +0000 UTC m=+2434.249635744" watchObservedRunningTime="2026-10-30 19:19:23.765774323 +0000 UTC m=+2434.255902144" Oct 30 19:19:23.816562 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.816520 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:23.830880 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.830839 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:23.836767 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.836701 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:23.900791 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:23.876730 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:24.108701 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.108613 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:24.426215 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.426182 5894 generic.go:356] "Generic (PLEG): container finished" podID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerID="22872da387db898771bf377c0134d75a87b8744dc5c42a8d8479c4eacdc5333a" exitCode=0 Oct 30 19:19:24.426560 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.426253 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" event={"ID":"4c72bcc9-7401-4b86-900d-558b768d8d16","Type":"ContainerDied","Data":"22872da387db898771bf377c0134d75a87b8744dc5c42a8d8479c4eacdc5333a"} Oct 30 19:19:24.604386 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.604060 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 19:19:24.625412 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.625381 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-tmp-dir\") pod \"4c72bcc9-7401-4b86-900d-558b768d8d16\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " Oct 30 19:19:24.625626 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.625422 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-dshm\") pod \"4c72bcc9-7401-4b86-900d-558b768d8d16\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " Oct 30 19:19:24.625626 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.625462 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-home\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-home\") pod \"4c72bcc9-7401-4b86-900d-558b768d8d16\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " Oct 30 19:19:24.625626 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.625551 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-model-cache\") pod \"4c72bcc9-7401-4b86-900d-558b768d8d16\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " Oct 30 19:19:24.625811 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.625646 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4c72bcc9-7401-4b86-900d-558b768d8d16-kube-api-access-lhz7w\" (UniqueName: \"kubernetes.io/projected/4c72bcc9-7401-4b86-900d-558b768d8d16-kube-api-access-lhz7w\") pod \"4c72bcc9-7401-4b86-900d-558b768d8d16\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " Oct 30 19:19:24.625811 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.625723 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4c72bcc9-7401-4b86-900d-558b768d8d16-tls-certs\" (UniqueName: \"kubernetes.io/secret/4c72bcc9-7401-4b86-900d-558b768d8d16-tls-certs\") pod \"4c72bcc9-7401-4b86-900d-558b768d8d16\" (UID: \"4c72bcc9-7401-4b86-900d-558b768d8d16\") " Oct 30 19:19:24.684655 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.626626 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-tmp-dir" pod "4c72bcc9-7401-4b86-900d-558b768d8d16" (UID: "4c72bcc9-7401-4b86-900d-558b768d8d16"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:19:24.684655 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.630653 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-home" pod "4c72bcc9-7401-4b86-900d-558b768d8d16" (UID: "4c72bcc9-7401-4b86-900d-558b768d8d16"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:19:24.684655 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.636653 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-model-cache" pod "4c72bcc9-7401-4b86-900d-558b768d8d16" (UID: "4c72bcc9-7401-4b86-900d-558b768d8d16"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:19:24.707933 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.707881 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c72bcc9-7401-4b86-900d-558b768d8d16-kube-api-access-lhz7w" pod "4c72bcc9-7401-4b86-900d-558b768d8d16" (UID: "4c72bcc9-7401-4b86-900d-558b768d8d16"). InnerVolumeSpecName "kube-api-access-lhz7w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:19:24.708445 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.708423 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c72bcc9-7401-4b86-900d-558b768d8d16-tls-certs" pod "4c72bcc9-7401-4b86-900d-558b768d8d16" (UID: "4c72bcc9-7401-4b86-900d-558b768d8d16"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:19:24.708945 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.708925 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-dshm" pod "4c72bcc9-7401-4b86-900d-558b768d8d16" (UID: "4c72bcc9-7401-4b86-900d-558b768d8d16"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:19:24.726771 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.726657 5894 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-model-cache\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:19:24.726771 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.726688 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhz7w\" (UniqueName: \"kubernetes.io/projected/4c72bcc9-7401-4b86-900d-558b768d8d16-kube-api-access-lhz7w\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:19:24.726771 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.726704 5894 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4c72bcc9-7401-4b86-900d-558b768d8d16-tls-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:19:24.726771 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.726717 5894 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-tmp-dir\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:19:24.726771 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.726730 5894 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-dshm\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:19:24.726771 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.726742 5894 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4c72bcc9-7401-4b86-900d-558b768d8d16-home\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:19:24.816180 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.816152 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-w22dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body= Oct 30 19:19:24.816892 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.816628 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" podUID="010f73a2-9a20-49a6-8d88-079a27465e78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" Oct 30 19:19:24.883612 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.858682 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-7z7fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body= Oct 30 19:19:24.883612 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.858741 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" podUID="75f81dfd-86e7-4532-a20c-42423f908903" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" Oct 30 19:19:24.883612 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.858826 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-nvv4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body= Oct 30 19:19:24.883612 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:24.858849 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" podUID="3fec8f46-c372-49cc-8c0a-9cb885f465be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" Oct 30 19:19:25.485605 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:25.477211 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" Oct 30 19:19:25.485605 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:25.481638 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn" event={"ID":"4c72bcc9-7401-4b86-900d-558b768d8d16","Type":"ContainerDied","Data":"2640ea3a8603b1b65e86f6a5d534452e45a603fdcad05e7ce911f1cbed69c808"} Oct 30 19:19:25.485605 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:25.481701 5894 scope.go:122] "RemoveContainer" containerID="22872da387db898771bf377c0134d75a87b8744dc5c42a8d8479c4eacdc5333a" Oct 30 19:19:25.723200 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:25.723168 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn"] Oct 30 19:19:25.723380 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:25.723213 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w5-611053/distinct-2-w5-611053-kserve-7dfc87b6dc-g2qgn"] Oct 30 19:19:25.794318 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:25.794225 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:25.814535 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:25.814497 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-w22dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body= Oct 30 19:19:25.814686 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:25.814563 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" podUID="010f73a2-9a20-49a6-8d88-079a27465e78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" Oct 30 19:19:25.835678 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:25.834914 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-7z7fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body= Oct 30 19:19:25.835678 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:25.834969 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" podUID="75f81dfd-86e7-4532-a20c-42423f908903" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" Oct 30 19:19:25.852606 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:25.847394 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-nvv4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body= Oct 30 19:19:25.852606 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:25.847444 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" podUID="3fec8f46-c372-49cc-8c0a-9cb885f465be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" Oct 30 19:19:26.815332 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:26.815298 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-w22dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body= Oct 30 19:19:26.815546 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:26.815517 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" podUID="010f73a2-9a20-49a6-8d88-079a27465e78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" Oct 30 19:19:26.849962 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:26.849925 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-7z7fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body= Oct 30 19:19:26.850535 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:26.850501 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" podUID="75f81dfd-86e7-4532-a20c-42423f908903" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" Oct 30 19:19:26.850718 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:26.850686 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-nvv4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body= Oct 30 19:19:26.850918 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:26.850894 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" podUID="3fec8f46-c372-49cc-8c0a-9cb885f465be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" Oct 30 19:19:27.096216 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:27.096111 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:27.682597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:27.682531 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" path="/var/lib/kubelet/pods/4c72bcc9-7401-4b86-900d-558b768d8d16/volumes" Oct 30 19:19:27.815243 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:27.815203 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-w22dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body= Oct 30 19:19:27.815442 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:27.815261 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" podUID="010f73a2-9a20-49a6-8d88-079a27465e78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" Oct 30 19:19:27.831607 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:27.830916 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-7z7fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body= Oct 30 19:19:27.831607 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:27.830988 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" podUID="75f81dfd-86e7-4532-a20c-42423f908903" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" Oct 30 19:19:27.836942 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:27.836907 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-nvv4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body= Oct 30 19:19:27.837166 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:27.837138 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" podUID="3fec8f46-c372-49cc-8c0a-9cb885f465be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" Oct 30 19:19:28.814813 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:28.814765 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-w22dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body= Oct 30 19:19:28.815342 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:28.814828 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" podUID="010f73a2-9a20-49a6-8d88-079a27465e78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" Oct 30 19:19:28.831047 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:28.831016 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-7z7fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body= Oct 30 19:19:28.831299 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:28.831273 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" podUID="75f81dfd-86e7-4532-a20c-42423f908903" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" Oct 30 19:19:28.842790 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:28.842680 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-nvv4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body= Oct 30 19:19:28.842790 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:28.842735 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" podUID="3fec8f46-c372-49cc-8c0a-9cb885f465be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" Oct 30 19:19:29.814921 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:29.814882 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-w22dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body= Oct 30 19:19:29.815402 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:29.814943 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" podUID="010f73a2-9a20-49a6-8d88-079a27465e78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" Oct 30 19:19:29.830895 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:29.830858 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-7z7fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body= Oct 30 19:19:29.831032 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:29.830919 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" podUID="75f81dfd-86e7-4532-a20c-42423f908903" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" Oct 30 19:19:29.837186 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:29.837159 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-nvv4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body= Oct 30 19:19:29.837276 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:29.837210 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" podUID="3fec8f46-c372-49cc-8c0a-9cb885f465be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" Oct 30 19:19:29.970678 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:29.970627 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:30.170984 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:30.170950 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:30.814543 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:30.814499 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-w22dg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" start-of-body= Oct 30 19:19:30.814914 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:30.814631 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" podUID="010f73a2-9a20-49a6-8d88-079a27465e78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": dial tcp 10.132.0.126:15021: connect: connection refused" Oct 30 19:19:30.830894 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:30.830861 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-7z7fl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" start-of-body= Oct 30 19:19:30.831233 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:30.830923 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" podUID="75f81dfd-86e7-4532-a20c-42423f908903" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": dial tcp 10.132.0.127:15021: connect: connection refused" Oct 30 19:19:30.836971 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:30.836931 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-nvv4s container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" start-of-body= Oct 30 19:19:30.837113 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:30.836978 5894 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" podUID="3fec8f46-c372-49cc-8c0a-9cb885f465be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": dial tcp 10.132.0.128:15021: connect: connection refused" Oct 30 19:19:32.428376 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:32.428343 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:32.498778 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:32.498732 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:32.503237 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:32.503066 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:32.504973 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:32.504953 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" Oct 30 19:19:32.517982 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:32.517068 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:32.522149 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:32.518393 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:32.522322 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:32.522281 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" Oct 30 19:19:32.546704 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:32.546674 5894 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:32.547201 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:32.547125 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:32.549147 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:32.549122 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" Oct 30 19:19:37.089679 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:37.089630 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:37.271347 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:37.269290 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:37.445743 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:37.445708 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:38.962052 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:38.962018 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:41.881469 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:41.881433 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:42.189257 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:42.189219 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:45.693073 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:45.693038 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:45.886342 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:45.886310 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:46.926679 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:46.926650 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:47.108864 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:47.108827 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:49.380010 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:49.379979 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:49.607169 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:49.607123 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:50.117966 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:19:50.117931 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:19:51.059977 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.059939 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt"] Oct 30 19:19:51.060413 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.060222 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" podUID="9faa41c5-f9c8-4355-a8ce-d5c44409e061" containerName="maas-api" containerID="cri-o://07a3d94fa8a145ae30e4899a1fd096db05312939e005fd9f70b7e3931370d22d" gracePeriod=30 Oct 30 19:19:51.203373 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.203316 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:51.356389 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.356354 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" Oct 30 19:19:51.416756 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.416723 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9faa41c5-f9c8-4355-a8ce-d5c44409e061-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9faa41c5-f9c8-4355-a8ce-d5c44409e061-maas-api-tls\") pod \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\" (UID: \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\") " Oct 30 19:19:51.416937 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.416839 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9faa41c5-f9c8-4355-a8ce-d5c44409e061-metrics-certs\" (UniqueName: \"kubernetes.io/secret/9faa41c5-f9c8-4355-a8ce-d5c44409e061-metrics-certs\") pod \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\" (UID: \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\") " Oct 30 19:19:51.416937 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.416874 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9faa41c5-f9c8-4355-a8ce-d5c44409e061-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9faa41c5-f9c8-4355-a8ce-d5c44409e061-openshift-service-ca\") pod \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\" (UID: \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\") " Oct 30 19:19:51.417072 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.416946 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9faa41c5-f9c8-4355-a8ce-d5c44409e061-kube-api-access-qjlf5\" (UniqueName: \"kubernetes.io/projected/9faa41c5-f9c8-4355-a8ce-d5c44409e061-kube-api-access-qjlf5\") pod \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\" (UID: \"9faa41c5-f9c8-4355-a8ce-d5c44409e061\") " Oct 30 19:19:51.417337 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.417306 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9faa41c5-f9c8-4355-a8ce-d5c44409e061-openshift-service-ca" pod "9faa41c5-f9c8-4355-a8ce-d5c44409e061" (UID: "9faa41c5-f9c8-4355-a8ce-d5c44409e061"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:19:51.419101 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.419074 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9faa41c5-f9c8-4355-a8ce-d5c44409e061-maas-api-tls" pod "9faa41c5-f9c8-4355-a8ce-d5c44409e061" (UID: "9faa41c5-f9c8-4355-a8ce-d5c44409e061"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:19:51.419211 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.419162 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9faa41c5-f9c8-4355-a8ce-d5c44409e061-metrics-certs" pod "9faa41c5-f9c8-4355-a8ce-d5c44409e061" (UID: "9faa41c5-f9c8-4355-a8ce-d5c44409e061"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:19:51.419489 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.419465 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9faa41c5-f9c8-4355-a8ce-d5c44409e061-kube-api-access-qjlf5" pod "9faa41c5-f9c8-4355-a8ce-d5c44409e061" (UID: "9faa41c5-f9c8-4355-a8ce-d5c44409e061"). InnerVolumeSpecName "kube-api-access-qjlf5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:19:51.518795 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.518766 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9faa41c5-f9c8-4355-a8ce-d5c44409e061-maas-api-tls\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:19:51.518795 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.518791 5894 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9faa41c5-f9c8-4355-a8ce-d5c44409e061-metrics-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:19:51.518795 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.518801 5894 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/9faa41c5-f9c8-4355-a8ce-d5c44409e061-openshift-service-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:19:51.519023 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.518811 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjlf5\" (UniqueName: \"kubernetes.io/projected/9faa41c5-f9c8-4355-a8ce-d5c44409e061-kube-api-access-qjlf5\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:19:51.671917 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.671879 5894 generic.go:356] "Generic (PLEG): container finished" podID="9faa41c5-f9c8-4355-a8ce-d5c44409e061" containerID="07a3d94fa8a145ae30e4899a1fd096db05312939e005fd9f70b7e3931370d22d" exitCode=0 Oct 30 19:19:51.672128 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.671924 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" Oct 30 19:19:51.672128 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.671965 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" event={"ID":"9faa41c5-f9c8-4355-a8ce-d5c44409e061","Type":"ContainerDied","Data":"07a3d94fa8a145ae30e4899a1fd096db05312939e005fd9f70b7e3931370d22d"} Oct 30 19:19:51.672128 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.672030 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt" event={"ID":"9faa41c5-f9c8-4355-a8ce-d5c44409e061","Type":"ContainerDied","Data":"c56d84d1fe931adc019032ad5830c5a5c4d4229c8b47f72a2371389b579606fe"} Oct 30 19:19:51.672128 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.672061 5894 scope.go:122] "RemoveContainer" containerID="07a3d94fa8a145ae30e4899a1fd096db05312939e005fd9f70b7e3931370d22d" Oct 30 19:19:51.683543 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.683524 5894 scope.go:122] "RemoveContainer" containerID="07a3d94fa8a145ae30e4899a1fd096db05312939e005fd9f70b7e3931370d22d" Oct 30 19:19:51.683942 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:19:51.683918 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07a3d94fa8a145ae30e4899a1fd096db05312939e005fd9f70b7e3931370d22d\": container with ID starting with 07a3d94fa8a145ae30e4899a1fd096db05312939e005fd9f70b7e3931370d22d not found: ID does not exist" containerID="07a3d94fa8a145ae30e4899a1fd096db05312939e005fd9f70b7e3931370d22d" Oct 30 19:19:51.684016 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.683955 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07a3d94fa8a145ae30e4899a1fd096db05312939e005fd9f70b7e3931370d22d"} err="failed to get container status \"07a3d94fa8a145ae30e4899a1fd096db05312939e005fd9f70b7e3931370d22d\": rpc error: code = NotFound desc = could not find container \"07a3d94fa8a145ae30e4899a1fd096db05312939e005fd9f70b7e3931370d22d\": container with ID starting with 07a3d94fa8a145ae30e4899a1fd096db05312939e005fd9f70b7e3931370d22d not found: ID does not exist" Oct 30 19:19:51.839550 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.839515 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt"] Oct 30 19:19:51.844602 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:51.844561 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w3-732a3f-767d646f7-vnxxt"] Oct 30 19:19:52.804177 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:52.804112 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:52.973797 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:52.973768 5894 scope.go:122] "RemoveContainer" containerID="3694dbd41ad629bf0d2638e0870bd49a522215392fd4a54a9e1aee6257ed781d" Oct 30 19:19:53.081429 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:53.081343 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:53.324740 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:53.324680 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:53.649737 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:53.649703 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9faa41c5-f9c8-4355-a8ce-d5c44409e061" path="/var/lib/kubelet/pods/9faa41c5-f9c8-4355-a8ce-d5c44409e061/volumes" Oct 30 19:19:53.715492 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:53.715458 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:57.970606 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:57.970546 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:59.787163 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:59.784267 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:19:59.980522 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:19:59.980486 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:00.063604 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:00.063505 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw"] Oct 30 19:20:00.063800 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:00.063771 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" podUID="8db11496-7f7c-49f1-ba40-08248a5b2db2" containerName="istio-proxy" containerID="cri-o://06aa1bf9f0975c86014627242a56ca2116df6e9bbb07afbca7bcd409907588d2" gracePeriod=30 Oct 30 19:20:00.180714 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:00.176621 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:00.564784 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:00.564721 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:02.777562 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:02.777527 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:05.324508 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.324476 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:20:05.479661 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.479625 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-token\" (UniqueName: \"kubernetes.io/projected/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-token\") pod \"8db11496-7f7c-49f1-ba40-08248a5b2db2\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " Oct 30 19:20:05.479880 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.479712 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/8db11496-7f7c-49f1-ba40-08248a5b2db2-kube-api-access-79fpd\" (UniqueName: \"kubernetes.io/projected/8db11496-7f7c-49f1-ba40-08248a5b2db2-kube-api-access-79fpd\") pod \"8db11496-7f7c-49f1-ba40-08248a5b2db2\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " Oct 30 19:20:05.479880 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.479752 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-credential-socket\") pod \"8db11496-7f7c-49f1-ba40-08248a5b2db2\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " Oct 30 19:20:05.479880 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.479777 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-data\") pod \"8db11496-7f7c-49f1-ba40-08248a5b2db2\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " Oct 30 19:20:05.479880 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.479806 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-workload-socket\") pod \"8db11496-7f7c-49f1-ba40-08248a5b2db2\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " Oct 30 19:20:05.480114 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.479879 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-podinfo\") pod \"8db11496-7f7c-49f1-ba40-08248a5b2db2\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " Oct 30 19:20:05.480114 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.479906 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/8db11496-7f7c-49f1-ba40-08248a5b2db2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8db11496-7f7c-49f1-ba40-08248a5b2db2-istiod-ca-cert\") pod \"8db11496-7f7c-49f1-ba40-08248a5b2db2\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " Oct 30 19:20:05.480114 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.479981 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-envoy\") pod \"8db11496-7f7c-49f1-ba40-08248a5b2db2\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " Oct 30 19:20:05.480114 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.480019 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-workload-certs\") pod \"8db11496-7f7c-49f1-ba40-08248a5b2db2\" (UID: \"8db11496-7f7c-49f1-ba40-08248a5b2db2\") " Oct 30 19:20:05.480114 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.480025 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-credential-socket" pod "8db11496-7f7c-49f1-ba40-08248a5b2db2" (UID: "8db11496-7f7c-49f1-ba40-08248a5b2db2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:05.480385 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.480109 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-data" pod "8db11496-7f7c-49f1-ba40-08248a5b2db2" (UID: "8db11496-7f7c-49f1-ba40-08248a5b2db2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:05.480470 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.480438 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.480470 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.480417 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8db11496-7f7c-49f1-ba40-08248a5b2db2-istiod-ca-cert" pod "8db11496-7f7c-49f1-ba40-08248a5b2db2" (UID: "8db11496-7f7c-49f1-ba40-08248a5b2db2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:20:05.480609 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.480461 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.480745 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.480701 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-workload-certs" pod "8db11496-7f7c-49f1-ba40-08248a5b2db2" (UID: "8db11496-7f7c-49f1-ba40-08248a5b2db2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:05.480827 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.480775 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-workload-socket" pod "8db11496-7f7c-49f1-ba40-08248a5b2db2" (UID: "8db11496-7f7c-49f1-ba40-08248a5b2db2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:05.482108 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.482085 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8db11496-7f7c-49f1-ba40-08248a5b2db2-kube-api-access-79fpd" pod "8db11496-7f7c-49f1-ba40-08248a5b2db2" (UID: "8db11496-7f7c-49f1-ba40-08248a5b2db2"). InnerVolumeSpecName "kube-api-access-79fpd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:20:05.482252 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.482233 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-envoy" pod "8db11496-7f7c-49f1-ba40-08248a5b2db2" (UID: "8db11496-7f7c-49f1-ba40-08248a5b2db2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:05.482393 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.482376 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-token" pod "8db11496-7f7c-49f1-ba40-08248a5b2db2" (UID: "8db11496-7f7c-49f1-ba40-08248a5b2db2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:20:05.482568 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.482376 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-podinfo" pod "8db11496-7f7c-49f1-ba40-08248a5b2db2" (UID: "8db11496-7f7c-49f1-ba40-08248a5b2db2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:20:05.485191 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.485171 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6"] Oct 30 19:20:05.538649 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.538615 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8"] Oct 30 19:20:05.581586 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.581543 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.581586 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.581569 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.581586 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.581595 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/8db11496-7f7c-49f1-ba40-08248a5b2db2-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.581841 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.581603 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.581841 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.581613 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/8db11496-7f7c-49f1-ba40-08248a5b2db2-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.581841 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.581621 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8db11496-7f7c-49f1-ba40-08248a5b2db2-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.581841 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.581631 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79fpd\" (UniqueName: \"kubernetes.io/projected/8db11496-7f7c-49f1-ba40-08248a5b2db2-kube-api-access-79fpd\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:05.740039 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.739953 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:05.751699 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.751663 5894 generic.go:356] "Generic (PLEG): container finished" podID="8db11496-7f7c-49f1-ba40-08248a5b2db2" containerID="06aa1bf9f0975c86014627242a56ca2116df6e9bbb07afbca7bcd409907588d2" exitCode=0 Oct 30 19:20:05.751881 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.751708 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" event={"ID":"8db11496-7f7c-49f1-ba40-08248a5b2db2","Type":"ContainerDied","Data":"06aa1bf9f0975c86014627242a56ca2116df6e9bbb07afbca7bcd409907588d2"} Oct 30 19:20:05.751881 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.751714 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" Oct 30 19:20:05.751881 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.751747 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw" event={"ID":"8db11496-7f7c-49f1-ba40-08248a5b2db2","Type":"ContainerDied","Data":"20cd7f7b02502520a9e2c7a09760f45ad8fb1ed8d0ab86b6bd70b92fb7fce402"} Oct 30 19:20:05.751881 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.751769 5894 scope.go:122] "RemoveContainer" containerID="06aa1bf9f0975c86014627242a56ca2116df6e9bbb07afbca7bcd409907588d2" Oct 30 19:20:05.761051 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.761027 5894 scope.go:122] "RemoveContainer" containerID="55a641a7e5e99592f04a6527fa9239ffdc56443ae313778feaa06dd679f9ef81" Oct 30 19:20:05.769875 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.769858 5894 scope.go:122] "RemoveContainer" containerID="06aa1bf9f0975c86014627242a56ca2116df6e9bbb07afbca7bcd409907588d2" Oct 30 19:20:05.770126 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:20:05.770106 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"06aa1bf9f0975c86014627242a56ca2116df6e9bbb07afbca7bcd409907588d2\": container with ID starting with 06aa1bf9f0975c86014627242a56ca2116df6e9bbb07afbca7bcd409907588d2 not found: ID does not exist" containerID="06aa1bf9f0975c86014627242a56ca2116df6e9bbb07afbca7bcd409907588d2" Oct 30 19:20:05.770172 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.770136 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"06aa1bf9f0975c86014627242a56ca2116df6e9bbb07afbca7bcd409907588d2"} err="failed to get container status \"06aa1bf9f0975c86014627242a56ca2116df6e9bbb07afbca7bcd409907588d2\": rpc error: code = NotFound desc = could not find container \"06aa1bf9f0975c86014627242a56ca2116df6e9bbb07afbca7bcd409907588d2\": container with ID starting with 06aa1bf9f0975c86014627242a56ca2116df6e9bbb07afbca7bcd409907588d2 not found: ID does not exist" Oct 30 19:20:05.770172 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.770152 5894 scope.go:122] "RemoveContainer" containerID="55a641a7e5e99592f04a6527fa9239ffdc56443ae313778feaa06dd679f9ef81" Oct 30 19:20:05.770369 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:20:05.770353 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"55a641a7e5e99592f04a6527fa9239ffdc56443ae313778feaa06dd679f9ef81\": container with ID starting with 55a641a7e5e99592f04a6527fa9239ffdc56443ae313778feaa06dd679f9ef81 not found: ID does not exist" containerID="55a641a7e5e99592f04a6527fa9239ffdc56443ae313778feaa06dd679f9ef81" Oct 30 19:20:05.770411 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.770374 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"55a641a7e5e99592f04a6527fa9239ffdc56443ae313778feaa06dd679f9ef81"} err="failed to get container status \"55a641a7e5e99592f04a6527fa9239ffdc56443ae313778feaa06dd679f9ef81\": rpc error: code = NotFound desc = could not find container \"55a641a7e5e99592f04a6527fa9239ffdc56443ae313778feaa06dd679f9ef81\": container with ID starting with 55a641a7e5e99592f04a6527fa9239ffdc56443ae313778feaa06dd679f9ef81 not found: ID does not exist" Oct 30 19:20:05.918725 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.918691 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw"] Oct 30 19:20:05.923881 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:05.923855 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w3-732a3f-openshift-default-655956c6c5-sm5pw"] Oct 30 19:20:07.649512 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:07.649479 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="8db11496-7f7c-49f1-ba40-08248a5b2db2" path="/var/lib/kubelet/pods/8db11496-7f7c-49f1-ba40-08248a5b2db2/volumes" Oct 30 19:20:07.909095 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:07.909018 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:12.631095 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:12.631060 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:12.800818 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:12.800788 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:16.658784 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:16.658754 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq"] Oct 30 19:20:16.659285 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:16.658958 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" podUID="bffc276a-bd01-4f6f-a9f2-252955fa734a" containerName="maas-api" containerID="cri-o://8805ca0740c8dfdd3dc91e19186a9bbdad51ddcedfac96c3a080125890bcf4ea" gracePeriod=30 Oct 30 19:20:16.800951 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:16.800919 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:16.818605 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:16.818541 5894 generic.go:356] "Generic (PLEG): container finished" podID="bffc276a-bd01-4f6f-a9f2-252955fa734a" containerID="8805ca0740c8dfdd3dc91e19186a9bbdad51ddcedfac96c3a080125890bcf4ea" exitCode=0 Oct 30 19:20:16.818766 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:16.818604 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" event={"ID":"bffc276a-bd01-4f6f-a9f2-252955fa734a","Type":"ContainerDied","Data":"8805ca0740c8dfdd3dc91e19186a9bbdad51ddcedfac96c3a080125890bcf4ea"} Oct 30 19:20:16.980499 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:16.980471 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" Oct 30 19:20:17.007327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:17.007291 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bffc276a-bd01-4f6f-a9f2-252955fa734a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/bffc276a-bd01-4f6f-a9f2-252955fa734a-metrics-certs\") pod \"bffc276a-bd01-4f6f-a9f2-252955fa734a\" (UID: \"bffc276a-bd01-4f6f-a9f2-252955fa734a\") " Oct 30 19:20:17.007470 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:17.007334 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/bffc276a-bd01-4f6f-a9f2-252955fa734a-kube-api-access-mwdz8\" (UniqueName: \"kubernetes.io/projected/bffc276a-bd01-4f6f-a9f2-252955fa734a-kube-api-access-mwdz8\") pod \"bffc276a-bd01-4f6f-a9f2-252955fa734a\" (UID: \"bffc276a-bd01-4f6f-a9f2-252955fa734a\") " Oct 30 19:20:17.007542 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:17.007518 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/bffc276a-bd01-4f6f-a9f2-252955fa734a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bffc276a-bd01-4f6f-a9f2-252955fa734a-maas-api-tls\") pod \"bffc276a-bd01-4f6f-a9f2-252955fa734a\" (UID: \"bffc276a-bd01-4f6f-a9f2-252955fa734a\") " Oct 30 19:20:17.007647 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:17.007623 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/bffc276a-bd01-4f6f-a9f2-252955fa734a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bffc276a-bd01-4f6f-a9f2-252955fa734a-openshift-service-ca\") pod \"bffc276a-bd01-4f6f-a9f2-252955fa734a\" (UID: \"bffc276a-bd01-4f6f-a9f2-252955fa734a\") " Oct 30 19:20:17.008002 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:17.007975 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bffc276a-bd01-4f6f-a9f2-252955fa734a-openshift-service-ca" pod "bffc276a-bd01-4f6f-a9f2-252955fa734a" (UID: "bffc276a-bd01-4f6f-a9f2-252955fa734a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:20:17.009857 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:17.009835 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bffc276a-bd01-4f6f-a9f2-252955fa734a-maas-api-tls" pod "bffc276a-bd01-4f6f-a9f2-252955fa734a" (UID: "bffc276a-bd01-4f6f-a9f2-252955fa734a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:20:17.010045 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:17.010016 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bffc276a-bd01-4f6f-a9f2-252955fa734a-metrics-certs" pod "bffc276a-bd01-4f6f-a9f2-252955fa734a" (UID: "bffc276a-bd01-4f6f-a9f2-252955fa734a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:20:17.010124 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:17.010046 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bffc276a-bd01-4f6f-a9f2-252955fa734a-kube-api-access-mwdz8" pod "bffc276a-bd01-4f6f-a9f2-252955fa734a" (UID: "bffc276a-bd01-4f6f-a9f2-252955fa734a"). InnerVolumeSpecName "kube-api-access-mwdz8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:20:17.109461 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:17.109427 5894 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bffc276a-bd01-4f6f-a9f2-252955fa734a-metrics-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:17.109461 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:17.109457 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwdz8\" (UniqueName: \"kubernetes.io/projected/bffc276a-bd01-4f6f-a9f2-252955fa734a-kube-api-access-mwdz8\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:17.109461 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:17.109468 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/bffc276a-bd01-4f6f-a9f2-252955fa734a-maas-api-tls\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:17.109733 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:17.109477 5894 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/bffc276a-bd01-4f6f-a9f2-252955fa734a-openshift-service-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:17.826860 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:17.826830 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" Oct 30 19:20:17.826860 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:17.826844 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq" event={"ID":"bffc276a-bd01-4f6f-a9f2-252955fa734a","Type":"ContainerDied","Data":"d9d81c9e64988aa38996f915c20b623133b5316e2504c35336b829f7b4c9ac03"} Oct 30 19:20:17.827305 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:17.826892 5894 scope.go:122] "RemoveContainer" containerID="8805ca0740c8dfdd3dc91e19186a9bbdad51ddcedfac96c3a080125890bcf4ea" Oct 30 19:20:17.997245 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:17.997167 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq"] Oct 30 19:20:18.046788 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:18.046757 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w5-611053-bfbdb889c-p9mlq"] Oct 30 19:20:18.288227 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:18.288138 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:18.572493 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:18.571771 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:18.784185 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:18.784148 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:19.204187 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:19.204145 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:19.650269 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:19.650235 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="bffc276a-bd01-4f6f-a9f2-252955fa734a" path="/var/lib/kubelet/pods/bffc276a-bd01-4f6f-a9f2-252955fa734a/volumes" Oct 30 19:20:20.117798 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:20:20.117712 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:20:20.505952 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.505898 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" containerID="cri-o://31d0600e78d1ec2f7e7be1e683fe284fbe0f7b6a64e08ec6b4e99dc8a5df39a1" gracePeriod=45 Oct 30 19:20:20.559593 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.559530 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" containerID="cri-o://c73922259a69a8a99a39c4820af9e0853c6bdea474a2880cd6249744d30b9231" gracePeriod=45 Oct 30 19:20:20.689659 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.689621 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" probeResult="failure" output="Get \"https://10.132.0.69:8000/ready\": dial tcp 10.132.0.69:8000: connect: connection refused" Oct 30 19:20:20.819001 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.818973 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 19:20:20.848101 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.848057 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-tls-certs\" (UniqueName: \"kubernetes.io/secret/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-tls-certs\") pod \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " Oct 30 19:20:20.848277 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.848121 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-kube-api-access-rlsz9\" (UniqueName: \"kubernetes.io/projected/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-kube-api-access-rlsz9\") pod \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " Oct 30 19:20:20.848277 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.848177 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-tmp-dir\") pod \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " Oct 30 19:20:20.848277 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.848228 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-dshm\") pod \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " Oct 30 19:20:20.848277 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.848256 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-home\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-home\") pod \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " Oct 30 19:20:20.848499 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.848292 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-model-cache\") pod \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\" (UID: \"e1e51799-7dbd-47ed-9b66-e4e355cc03b7\") " Oct 30 19:20:20.848887 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.848861 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-model-cache" pod "e1e51799-7dbd-47ed-9b66-e4e355cc03b7" (UID: "e1e51799-7dbd-47ed-9b66-e4e355cc03b7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:20.849072 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.849057 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-home" pod "e1e51799-7dbd-47ed-9b66-e4e355cc03b7" (UID: "e1e51799-7dbd-47ed-9b66-e4e355cc03b7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:20.849345 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.849317 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-tmp-dir" pod "e1e51799-7dbd-47ed-9b66-e4e355cc03b7" (UID: "e1e51799-7dbd-47ed-9b66-e4e355cc03b7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:20.851002 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.850969 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-tls-certs" pod "e1e51799-7dbd-47ed-9b66-e4e355cc03b7" (UID: "e1e51799-7dbd-47ed-9b66-e4e355cc03b7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:20:20.851194 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.851165 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-dshm" pod "e1e51799-7dbd-47ed-9b66-e4e355cc03b7" (UID: "e1e51799-7dbd-47ed-9b66-e4e355cc03b7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:20.851760 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.851732 5894 generic.go:356] "Generic (PLEG): container finished" podID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerID="31d0600e78d1ec2f7e7be1e683fe284fbe0f7b6a64e08ec6b4e99dc8a5df39a1" exitCode=0 Oct 30 19:20:20.851885 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.851866 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" event={"ID":"e1e51799-7dbd-47ed-9b66-e4e355cc03b7","Type":"ContainerDied","Data":"31d0600e78d1ec2f7e7be1e683fe284fbe0f7b6a64e08ec6b4e99dc8a5df39a1"} Oct 30 19:20:20.851977 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.851907 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" event={"ID":"e1e51799-7dbd-47ed-9b66-e4e355cc03b7","Type":"ContainerDied","Data":"96c128daca2cab662799714dce29bdaefd496c68939165289f0c123f83b75664"} Oct 30 19:20:20.851977 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.851932 5894 scope.go:122] "RemoveContainer" containerID="31d0600e78d1ec2f7e7be1e683fe284fbe0f7b6a64e08ec6b4e99dc8a5df39a1" Oct 30 19:20:20.852091 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.852056 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6" Oct 30 19:20:20.852320 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.852275 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-kube-api-access-rlsz9" pod "e1e51799-7dbd-47ed-9b66-e4e355cc03b7" (UID: "e1e51799-7dbd-47ed-9b66-e4e355cc03b7"). InnerVolumeSpecName "kube-api-access-rlsz9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:20:20.854055 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.854032 5894 generic.go:356] "Generic (PLEG): container finished" podID="5935a99f-0b26-4651-a937-d76762e3ce55" containerID="c73922259a69a8a99a39c4820af9e0853c6bdea474a2880cd6249744d30b9231" exitCode=0 Oct 30 19:20:20.854187 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.854115 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" event={"ID":"5935a99f-0b26-4651-a937-d76762e3ce55","Type":"ContainerDied","Data":"c73922259a69a8a99a39c4820af9e0853c6bdea474a2880cd6249744d30b9231"} Oct 30 19:20:20.867747 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.867730 5894 scope.go:122] "RemoveContainer" containerID="31d0600e78d1ec2f7e7be1e683fe284fbe0f7b6a64e08ec6b4e99dc8a5df39a1" Oct 30 19:20:20.868017 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:20:20.867995 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"31d0600e78d1ec2f7e7be1e683fe284fbe0f7b6a64e08ec6b4e99dc8a5df39a1\": container with ID starting with 31d0600e78d1ec2f7e7be1e683fe284fbe0f7b6a64e08ec6b4e99dc8a5df39a1 not found: ID does not exist" containerID="31d0600e78d1ec2f7e7be1e683fe284fbe0f7b6a64e08ec6b4e99dc8a5df39a1" Oct 30 19:20:20.868074 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.868026 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"31d0600e78d1ec2f7e7be1e683fe284fbe0f7b6a64e08ec6b4e99dc8a5df39a1"} err="failed to get container status \"31d0600e78d1ec2f7e7be1e683fe284fbe0f7b6a64e08ec6b4e99dc8a5df39a1\": rpc error: code = NotFound desc = could not find container \"31d0600e78d1ec2f7e7be1e683fe284fbe0f7b6a64e08ec6b4e99dc8a5df39a1\": container with ID starting with 31d0600e78d1ec2f7e7be1e683fe284fbe0f7b6a64e08ec6b4e99dc8a5df39a1 not found: ID does not exist" Oct 30 19:20:20.871615 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.871597 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 19:20:20.949474 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.949437 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-home\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-home\") pod \"5935a99f-0b26-4651-a937-d76762e3ce55\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " Oct 30 19:20:20.949677 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.949491 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-dshm\") pod \"5935a99f-0b26-4651-a937-d76762e3ce55\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " Oct 30 19:20:20.949677 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.949548 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-model-cache\") pod \"5935a99f-0b26-4651-a937-d76762e3ce55\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " Oct 30 19:20:20.949677 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.949654 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5935a99f-0b26-4651-a937-d76762e3ce55-tls-certs\" (UniqueName: \"kubernetes.io/secret/5935a99f-0b26-4651-a937-d76762e3ce55-tls-certs\") pod \"5935a99f-0b26-4651-a937-d76762e3ce55\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " Oct 30 19:20:20.949861 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.949688 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5935a99f-0b26-4651-a937-d76762e3ce55-kube-api-access-wrscf\" (UniqueName: \"kubernetes.io/projected/5935a99f-0b26-4651-a937-d76762e3ce55-kube-api-access-wrscf\") pod \"5935a99f-0b26-4651-a937-d76762e3ce55\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " Oct 30 19:20:20.949861 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.949695 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-home" pod "5935a99f-0b26-4651-a937-d76762e3ce55" (UID: "5935a99f-0b26-4651-a937-d76762e3ce55"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:20.949861 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.949805 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-tmp-dir\") pod \"5935a99f-0b26-4651-a937-d76762e3ce55\" (UID: \"5935a99f-0b26-4651-a937-d76762e3ce55\") " Oct 30 19:20:20.949861 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.949841 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-model-cache" pod "5935a99f-0b26-4651-a937-d76762e3ce55" (UID: "5935a99f-0b26-4651-a937-d76762e3ce55"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:20.950120 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.950103 5894 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-model-cache\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:20.950195 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.950127 5894 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-home\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:20.950195 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.950136 5894 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-model-cache\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:20.950195 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.950145 5894 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-tls-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:20.950195 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.950155 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rlsz9\" (UniqueName: \"kubernetes.io/projected/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-kube-api-access-rlsz9\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:20.950195 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.950163 5894 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-tmp-dir\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:20.950195 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.950171 5894 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-dshm\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:20.950195 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.950178 5894 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e1e51799-7dbd-47ed-9b66-e4e355cc03b7-home\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:20.950195 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.950100 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-tmp-dir" pod "5935a99f-0b26-4651-a937-d76762e3ce55" (UID: "5935a99f-0b26-4651-a937-d76762e3ce55"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:20.951697 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.951670 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-dshm" pod "5935a99f-0b26-4651-a937-d76762e3ce55" (UID: "5935a99f-0b26-4651-a937-d76762e3ce55"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:20.951847 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.951830 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5935a99f-0b26-4651-a937-d76762e3ce55-tls-certs" pod "5935a99f-0b26-4651-a937-d76762e3ce55" (UID: "5935a99f-0b26-4651-a937-d76762e3ce55"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:20:20.951927 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:20.951843 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5935a99f-0b26-4651-a937-d76762e3ce55-kube-api-access-wrscf" pod "5935a99f-0b26-4651-a937-d76762e3ce55" (UID: "5935a99f-0b26-4651-a937-d76762e3ce55"). InnerVolumeSpecName "kube-api-access-wrscf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:20:21.051495 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:21.051416 5894 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5935a99f-0b26-4651-a937-d76762e3ce55-tls-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:21.051495 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:21.051443 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrscf\" (UniqueName: \"kubernetes.io/projected/5935a99f-0b26-4651-a937-d76762e3ce55-kube-api-access-wrscf\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:21.051495 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:21.051453 5894 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-tmp-dir\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:21.051495 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:21.051462 5894 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5935a99f-0b26-4651-a937-d76762e3ce55-dshm\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:21.271679 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:21.271531 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6"] Oct 30 19:20:21.276134 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:21.276107 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-bc98e1/distinct-2-w3-bc98e1-kserve-6f645f86bf-jbkp6"] Oct 30 19:20:21.649305 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:21.649264 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" path="/var/lib/kubelet/pods/e1e51799-7dbd-47ed-9b66-e4e355cc03b7/volumes" Oct 30 19:20:21.863530 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:21.863463 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" Oct 30 19:20:21.863530 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:21.863482 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8" event={"ID":"5935a99f-0b26-4651-a937-d76762e3ce55","Type":"ContainerDied","Data":"ac4a3922c9dfacfa65193aae8723dcdefd98d9498cc934e1551228816bc7119b"} Oct 30 19:20:21.863530 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:21.863531 5894 scope.go:122] "RemoveContainer" containerID="c73922259a69a8a99a39c4820af9e0853c6bdea474a2880cd6249744d30b9231" Oct 30 19:20:22.030184 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:22.030151 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8"] Oct 30 19:20:22.079823 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:22.079792 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-bc98e1/premium-w3-bc98e1-kserve-5bd4b9f947-mmsq8"] Oct 30 19:20:23.648731 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:23.648698 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" path="/var/lib/kubelet/pods/5935a99f-0b26-4651-a937-d76762e3ce55/volumes" Oct 30 19:20:23.815373 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:23.815335 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:25.413186 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:25.413144 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:25.594981 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:25.594924 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:25.717701 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:25.717259 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x"] Oct 30 19:20:25.717701 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:25.717512 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" podUID="f247ccba-3633-4301-b8f2-41a0a8eab7ea" containerName="istio-proxy" containerID="cri-o://1b7c620cead080673b81823bedc5f6f6f14586c9d04119027b35dd7f68655b82" gracePeriod=30 Oct 30 19:20:25.814525 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:25.813899 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:26.188927 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:26.188891 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:28.316835 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:28.316803 5894 patch_prober.go:29] interesting pod/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:20:28.317229 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:28.316857 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" podUID="f247ccba-3633-4301-b8f2-41a0a8eab7ea" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:20:28.551304 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:28.551268 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:30.915685 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:30.915654 5894 generic.go:356] "Generic (PLEG): container finished" podID="f247ccba-3633-4301-b8f2-41a0a8eab7ea" containerID="1b7c620cead080673b81823bedc5f6f6f14586c9d04119027b35dd7f68655b82" exitCode=0 Oct 30 19:20:30.916052 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:30.915733 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" event={"ID":"f247ccba-3633-4301-b8f2-41a0a8eab7ea","Type":"ContainerDied","Data":"1b7c620cead080673b81823bedc5f6f6f14586c9d04119027b35dd7f68655b82"} Oct 30 19:20:30.981614 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:30.981563 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:20:31.080756 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.080674 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-token\" (UniqueName: \"kubernetes.io/projected/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-token\") pod \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " Oct 30 19:20:31.080756 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.080729 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istiod-ca-cert\") pod \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " Oct 30 19:20:31.080974 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.080760 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-workload-socket\") pod \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " Oct 30 19:20:31.080974 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.080823 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-workload-certs\") pod \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " Oct 30 19:20:31.080974 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.080879 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-credential-socket\") pod \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " Oct 30 19:20:31.080974 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.080938 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-envoy\") pod \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " Oct 30 19:20:31.080974 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.080962 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-data\") pod \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " Oct 30 19:20:31.081157 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.081002 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f247ccba-3633-4301-b8f2-41a0a8eab7ea-kube-api-access-4c7bl\" (UniqueName: \"kubernetes.io/projected/f247ccba-3633-4301-b8f2-41a0a8eab7ea-kube-api-access-4c7bl\") pod \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " Oct 30 19:20:31.081157 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.081109 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-podinfo\") pod \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\" (UID: \"f247ccba-3633-4301-b8f2-41a0a8eab7ea\") " Oct 30 19:20:31.081266 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.081149 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-workload-certs" pod "f247ccba-3633-4301-b8f2-41a0a8eab7ea" (UID: "f247ccba-3633-4301-b8f2-41a0a8eab7ea"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:31.081349 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.081239 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istiod-ca-cert" pod "f247ccba-3633-4301-b8f2-41a0a8eab7ea" (UID: "f247ccba-3633-4301-b8f2-41a0a8eab7ea"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:20:31.081417 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.081374 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-workload-socket" pod "f247ccba-3633-4301-b8f2-41a0a8eab7ea" (UID: "f247ccba-3633-4301-b8f2-41a0a8eab7ea"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:31.081417 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.081319 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-data" pod "f247ccba-3633-4301-b8f2-41a0a8eab7ea" (UID: "f247ccba-3633-4301-b8f2-41a0a8eab7ea"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:31.081417 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.081398 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-credential-socket" pod "f247ccba-3633-4301-b8f2-41a0a8eab7ea" (UID: "f247ccba-3633-4301-b8f2-41a0a8eab7ea"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:31.081670 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.081645 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:31.081758 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.081675 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:31.081758 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.081690 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:31.081758 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.081704 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:31.081758 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.081717 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:31.083104 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.083077 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-token" pod "f247ccba-3633-4301-b8f2-41a0a8eab7ea" (UID: "f247ccba-3633-4301-b8f2-41a0a8eab7ea"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:20:31.083213 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.083152 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-envoy" pod "f247ccba-3633-4301-b8f2-41a0a8eab7ea" (UID: "f247ccba-3633-4301-b8f2-41a0a8eab7ea"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:31.083361 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.083334 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-podinfo" pod "f247ccba-3633-4301-b8f2-41a0a8eab7ea" (UID: "f247ccba-3633-4301-b8f2-41a0a8eab7ea"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:20:31.083361 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.083333 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f247ccba-3633-4301-b8f2-41a0a8eab7ea-kube-api-access-4c7bl" pod "f247ccba-3633-4301-b8f2-41a0a8eab7ea" (UID: "f247ccba-3633-4301-b8f2-41a0a8eab7ea"). InnerVolumeSpecName "kube-api-access-4c7bl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:20:31.182520 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.182487 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:31.182520 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.182519 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4c7bl\" (UniqueName: \"kubernetes.io/projected/f247ccba-3633-4301-b8f2-41a0a8eab7ea-kube-api-access-4c7bl\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:31.182729 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.182532 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:31.182729 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.182541 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f247ccba-3633-4301-b8f2-41a0a8eab7ea-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:31.923569 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.923486 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" Oct 30 19:20:31.923569 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.923503 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x" event={"ID":"f247ccba-3633-4301-b8f2-41a0a8eab7ea","Type":"ContainerDied","Data":"c0bf8d3e7dfd8d51240f899347bf8afa8fbaebc605a4df391a0ff8fc902caf9e"} Oct 30 19:20:31.924031 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:31.923619 5894 scope.go:122] "RemoveContainer" containerID="1b7c620cead080673b81823bedc5f6f6f14586c9d04119027b35dd7f68655b82" Oct 30 19:20:32.143131 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:32.143099 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x"] Oct 30 19:20:32.192346 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:32.192282 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w5-611053-openshift-default-5cf969d899-bjn7x"] Oct 30 19:20:33.280126 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:33.280087 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:33.649123 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:33.649089 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f247ccba-3633-4301-b8f2-41a0a8eab7ea" path="/var/lib/kubelet/pods/f247ccba-3633-4301-b8f2-41a0a8eab7ea/volumes" Oct 30 19:20:37.392014 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:37.391976 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:38.970094 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:38.970037 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:39.316138 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:39.316044 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:39.510043 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:39.510009 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:39.922220 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:39.922178 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:42.411268 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:42.411230 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:42.593179 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:42.588325 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:43.695454 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:43.695410 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:44.353133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:44.353098 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:46.662669 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:46.662634 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:46.853106 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:46.853072 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:46.971934 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:46.971861 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn"] Oct 30 19:20:46.972084 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:46.972060 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" podUID="a37b4ddc-1a56-4ebe-a548-d2681502698a" containerName="istio-proxy" containerID="cri-o://c1be93fb4c492a5ee08bf28e7bfe36420a1f448b7c3bf5ae59c5fd27513f3b19" gracePeriod=30 Oct 30 19:20:47.054025 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:47.053986 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:47.623466 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:47.623433 5894 patch_prober.go:29] interesting pod/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:20:47.623712 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:47.623494 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" podUID="a37b4ddc-1a56-4ebe-a548-d2681502698a" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:20:49.067592 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:49.067540 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:49.469059 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:49.469026 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:50.117178 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:20:50.117140 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:20:52.234543 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.234517 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:20:52.315460 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.315392 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a37b4ddc-1a56-4ebe-a548-d2681502698a-kube-api-access-7nv8n\" (UniqueName: \"kubernetes.io/projected/a37b4ddc-1a56-4ebe-a548-d2681502698a-kube-api-access-7nv8n\") pod \"a37b4ddc-1a56-4ebe-a548-d2681502698a\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " Oct 30 19:20:52.315460 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.315425 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/a37b4ddc-1a56-4ebe-a548-d2681502698a-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a37b4ddc-1a56-4ebe-a548-d2681502698a-istiod-ca-cert\") pod \"a37b4ddc-1a56-4ebe-a548-d2681502698a\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " Oct 30 19:20:52.315460 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.315454 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-credential-socket\") pod \"a37b4ddc-1a56-4ebe-a548-d2681502698a\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " Oct 30 19:20:52.315707 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.315483 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-envoy\") pod \"a37b4ddc-1a56-4ebe-a548-d2681502698a\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " Oct 30 19:20:52.315707 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.315657 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-data\") pod \"a37b4ddc-1a56-4ebe-a548-d2681502698a\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " Oct 30 19:20:52.315801 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.315765 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-podinfo\") pod \"a37b4ddc-1a56-4ebe-a548-d2681502698a\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " Oct 30 19:20:52.315861 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.315828 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-workload-socket\") pod \"a37b4ddc-1a56-4ebe-a548-d2681502698a\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " Oct 30 19:20:52.315917 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.315864 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-token\" (UniqueName: \"kubernetes.io/projected/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-token\") pod \"a37b4ddc-1a56-4ebe-a548-d2681502698a\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " Oct 30 19:20:52.315976 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.315940 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-workload-certs\") pod \"a37b4ddc-1a56-4ebe-a548-d2681502698a\" (UID: \"a37b4ddc-1a56-4ebe-a548-d2681502698a\") " Oct 30 19:20:52.315976 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.315942 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-credential-socket" pod "a37b4ddc-1a56-4ebe-a548-d2681502698a" (UID: "a37b4ddc-1a56-4ebe-a548-d2681502698a"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:52.315976 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.315952 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-data" pod "a37b4ddc-1a56-4ebe-a548-d2681502698a" (UID: "a37b4ddc-1a56-4ebe-a548-d2681502698a"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:52.316147 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.315969 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a37b4ddc-1a56-4ebe-a548-d2681502698a-istiod-ca-cert" pod "a37b4ddc-1a56-4ebe-a548-d2681502698a" (UID: "a37b4ddc-1a56-4ebe-a548-d2681502698a"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:20:52.316209 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.316156 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-workload-socket" pod "a37b4ddc-1a56-4ebe-a548-d2681502698a" (UID: "a37b4ddc-1a56-4ebe-a548-d2681502698a"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:52.316209 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.316174 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-workload-certs" pod "a37b4ddc-1a56-4ebe-a548-d2681502698a" (UID: "a37b4ddc-1a56-4ebe-a548-d2681502698a"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:52.316398 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.316376 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:52.316477 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.316402 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:52.316477 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.316418 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a37b4ddc-1a56-4ebe-a548-d2681502698a-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:52.316477 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.316433 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:52.316477 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.316447 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:52.317834 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.317804 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-envoy" pod "a37b4ddc-1a56-4ebe-a548-d2681502698a" (UID: "a37b4ddc-1a56-4ebe-a548-d2681502698a"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:20:52.317959 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.317898 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a37b4ddc-1a56-4ebe-a548-d2681502698a-kube-api-access-7nv8n" pod "a37b4ddc-1a56-4ebe-a548-d2681502698a" (UID: "a37b4ddc-1a56-4ebe-a548-d2681502698a"). InnerVolumeSpecName "kube-api-access-7nv8n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:20:52.317959 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.317941 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-podinfo" pod "a37b4ddc-1a56-4ebe-a548-d2681502698a" (UID: "a37b4ddc-1a56-4ebe-a548-d2681502698a"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:20:52.318088 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.317974 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-token" pod "a37b4ddc-1a56-4ebe-a548-d2681502698a" (UID: "a37b4ddc-1a56-4ebe-a548-d2681502698a"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:20:52.417698 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.417661 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:52.417698 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.417699 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7nv8n\" (UniqueName: \"kubernetes.io/projected/a37b4ddc-1a56-4ebe-a548-d2681502698a-kube-api-access-7nv8n\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:52.417897 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.417714 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:52.417897 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:52.417726 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a37b4ddc-1a56-4ebe-a548-d2681502698a-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:20:53.005975 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:53.005946 5894 scope.go:122] "RemoveContainer" containerID="c1be93fb4c492a5ee08bf28e7bfe36420a1f448b7c3bf5ae59c5fd27513f3b19" Oct 30 19:20:53.039990 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:53.039963 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" event={"ID":"a37b4ddc-1a56-4ebe-a548-d2681502698a","Type":"ContainerDied","Data":"c1be93fb4c492a5ee08bf28e7bfe36420a1f448b7c3bf5ae59c5fd27513f3b19"} Oct 30 19:20:53.040066 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:53.039982 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" Oct 30 19:20:53.040066 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:53.040021 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn" event={"ID":"a37b4ddc-1a56-4ebe-a548-d2681502698a","Type":"ContainerDied","Data":"3e123210d6e2de690cb676edf42d109a178da02c332d5db50e0e49f7c749be9a"} Oct 30 19:20:53.321068 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:53.320973 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn"] Oct 30 19:20:53.325434 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:53.325408 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-bc98e1-openshift-default-5b4ff68f6c-2brpn"] Oct 30 19:20:53.649190 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:53.649157 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="a37b4ddc-1a56-4ebe-a548-d2681502698a" path="/var/lib/kubelet/pods/a37b4ddc-1a56-4ebe-a548-d2681502698a/volumes" Oct 30 19:20:53.824746 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:53.824710 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:20:58.563062 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:20:58.563028 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:01.249511 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:01.249435 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:03.374870 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:03.374771 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:08.075970 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:08.075882 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:08.248141 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:08.248109 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:12.841835 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:12.841800 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:14.867076 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:14.867034 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:17.507426 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:17.507363 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:20.117841 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:21:20.117801 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:21:22.165565 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:22.165524 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:26.478739 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:26.478684 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:26.809346 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:26.809240 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:27.110664 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:27.110559 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:27.290115 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:27.290081 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:29.901120 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:29.901085 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:30.104723 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:30.104689 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:31.525440 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:31.525344 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:36.275551 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:36.275511 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:41.118834 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:41.118797 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:43.373380 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:43.373345 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:45.919276 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:45.919237 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:50.117800 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:21:50.117759 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:21:50.975033 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:50.974992 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:53.050310 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:53.050275 5894 scope.go:122] "RemoveContainer" containerID="1ba9a60c1c344d73652c24a20bb96c6818a4469eb7fac33875bf5603305b1f88" Oct 30 19:21:55.029167 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:55.029136 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct"] Oct 30 19:21:55.037473 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:55.037446 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:55.133078 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:55.133047 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs"] Oct 30 19:21:55.452401 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:55.452365 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:55.641241 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:55.641209 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:56.782400 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:56.782352 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:59.376193 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:59.376159 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:21:59.660751 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:21:59.660711 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:00.030436 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:00.030352 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" podUID="1b395010-277d-42b3-b191-0e24f69156c1" containerName="payload-processing" containerID="cri-o://aa8be6fce6695580a0d73c25241124a922c7b7bb4b70f61e9796104c2b4b4d95" gracePeriod=25 Oct 30 19:22:00.133670 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:00.133628 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" podUID="eb9bc7c3-4527-4569-b896-b302d1bc5afc" containerName="payload-pre-processing" containerID="cri-o://898b42ca95974646e1b62dbe90c030f7d79e238ab17ddffb3ccf42171e6f6cf5" gracePeriod=25 Oct 30 19:22:00.321669 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:00.321565 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:01.704851 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:01.704811 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:06.260496 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:06.260455 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:06.461294 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:06.461241 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:06.580689 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:06.580609 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4"] Oct 30 19:22:06.580853 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:06.580827 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" podUID="b0479234-1db6-465f-9454-54d777b09538" containerName="istio-proxy" containerID="cri-o://d0f29a229fd44421812be3865231c2954a10a8d17e421382155f16aa9bd32f78" gracePeriod=30 Oct 30 19:22:06.647841 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:06.647803 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:07.695267 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:07.695224 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" podUID="eb9bc7c3-4527-4569-b896-b302d1bc5afc" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:22:08.702915 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:08.702879 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" podUID="1b395010-277d-42b3-b191-0e24f69156c1" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:22:10.224031 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:10.223993 5894 patch_prober.go:29] interesting pod/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Oct 30 19:22:10.224410 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:10.224048 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" podUID="b0479234-1db6-465f-9454-54d777b09538" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Oct 30 19:22:11.550645 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.550605 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:11.840830 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.840804 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:22:11.874973 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.874943 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/b0479234-1db6-465f-9454-54d777b09538-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0479234-1db6-465f-9454-54d777b09538-istiod-ca-cert\") pod \"b0479234-1db6-465f-9454-54d777b09538\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " Oct 30 19:22:11.875092 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.874977 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-credential-socket\") pod \"b0479234-1db6-465f-9454-54d777b09538\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " Oct 30 19:22:11.875092 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.875016 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-workload-certs\") pod \"b0479234-1db6-465f-9454-54d777b09538\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " Oct 30 19:22:11.875092 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.875043 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0479234-1db6-465f-9454-54d777b09538-kube-api-access-wg5r8\" (UniqueName: \"kubernetes.io/projected/b0479234-1db6-465f-9454-54d777b09538-kube-api-access-wg5r8\") pod \"b0479234-1db6-465f-9454-54d777b09538\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " Oct 30 19:22:11.875092 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.875070 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/b0479234-1db6-465f-9454-54d777b09538-istio-token\" (UniqueName: \"kubernetes.io/projected/b0479234-1db6-465f-9454-54d777b09538-istio-token\") pod \"b0479234-1db6-465f-9454-54d777b09538\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " Oct 30 19:22:11.875263 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.875205 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-istio-data\") pod \"b0479234-1db6-465f-9454-54d777b09538\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " Oct 30 19:22:11.875327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.875253 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-workload-certs" pod "b0479234-1db6-465f-9454-54d777b09538" (UID: "b0479234-1db6-465f-9454-54d777b09538"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:22:11.875327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.875280 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-istio-envoy\") pod \"b0479234-1db6-465f-9454-54d777b09538\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " Oct 30 19:22:11.875441 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.875325 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-workload-socket\") pod \"b0479234-1db6-465f-9454-54d777b09538\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " Oct 30 19:22:11.875441 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.875371 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/b0479234-1db6-465f-9454-54d777b09538-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0479234-1db6-465f-9454-54d777b09538-istio-podinfo\") pod \"b0479234-1db6-465f-9454-54d777b09538\" (UID: \"b0479234-1db6-465f-9454-54d777b09538\") " Oct 30 19:22:11.875441 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.875279 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-credential-socket" pod "b0479234-1db6-465f-9454-54d777b09538" (UID: "b0479234-1db6-465f-9454-54d777b09538"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:22:11.875441 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.875370 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b0479234-1db6-465f-9454-54d777b09538-istiod-ca-cert" pod "b0479234-1db6-465f-9454-54d777b09538" (UID: "b0479234-1db6-465f-9454-54d777b09538"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:22:11.875708 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.875537 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-istio-data" pod "b0479234-1db6-465f-9454-54d777b09538" (UID: "b0479234-1db6-465f-9454-54d777b09538"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:22:11.875767 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.875718 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-workload-socket" pod "b0479234-1db6-465f-9454-54d777b09538" (UID: "b0479234-1db6-465f-9454-54d777b09538"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:22:11.875824 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.875782 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:22:11.875824 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.875805 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:22:11.875934 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.875820 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b0479234-1db6-465f-9454-54d777b09538-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:22:11.875934 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.875837 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:22:11.877372 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.877343 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0479234-1db6-465f-9454-54d777b09538-istio-token" pod "b0479234-1db6-465f-9454-54d777b09538" (UID: "b0479234-1db6-465f-9454-54d777b09538"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:22:11.877372 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.877352 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b0479234-1db6-465f-9454-54d777b09538-kube-api-access-wg5r8" pod "b0479234-1db6-465f-9454-54d777b09538" (UID: "b0479234-1db6-465f-9454-54d777b09538"). InnerVolumeSpecName "kube-api-access-wg5r8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:22:11.877507 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.877408 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-istio-envoy" pod "b0479234-1db6-465f-9454-54d777b09538" (UID: "b0479234-1db6-465f-9454-54d777b09538"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:22:11.877507 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.877484 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b0479234-1db6-465f-9454-54d777b09538-istio-podinfo" pod "b0479234-1db6-465f-9454-54d777b09538" (UID: "b0479234-1db6-465f-9454-54d777b09538"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:22:11.977148 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.977116 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:22:11.977148 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.977146 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b0479234-1db6-465f-9454-54d777b09538-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:22:11.977330 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.977159 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b0479234-1db6-465f-9454-54d777b09538-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:22:11.977330 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.977169 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wg5r8\" (UniqueName: \"kubernetes.io/projected/b0479234-1db6-465f-9454-54d777b09538-kube-api-access-wg5r8\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:22:11.977330 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:11.977180 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b0479234-1db6-465f-9454-54d777b09538-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:22:12.477392 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:12.477354 5894 generic.go:356] "Generic (PLEG): container finished" podID="b0479234-1db6-465f-9454-54d777b09538" containerID="d0f29a229fd44421812be3865231c2954a10a8d17e421382155f16aa9bd32f78" exitCode=0 Oct 30 19:22:12.477595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:12.477399 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" Oct 30 19:22:12.477595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:12.477441 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" event={"ID":"b0479234-1db6-465f-9454-54d777b09538","Type":"ContainerDied","Data":"d0f29a229fd44421812be3865231c2954a10a8d17e421382155f16aa9bd32f78"} Oct 30 19:22:12.477595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:12.477503 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4" event={"ID":"b0479234-1db6-465f-9454-54d777b09538","Type":"ContainerDied","Data":"624ae13ac7fe2da5306a2a5737608147a36c6c13215d1e8a6dafceb93b5700db"} Oct 30 19:22:12.477595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:12.477526 5894 scope.go:122] "RemoveContainer" containerID="d0f29a229fd44421812be3865231c2954a10a8d17e421382155f16aa9bd32f78" Oct 30 19:22:12.487547 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:12.487531 5894 scope.go:122] "RemoveContainer" containerID="cdc0d40f50f26e4e40419f58a5a929522a0e258cae6deefd5c7127e887a48cc7" Oct 30 19:22:12.495736 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:12.495719 5894 scope.go:122] "RemoveContainer" containerID="d0f29a229fd44421812be3865231c2954a10a8d17e421382155f16aa9bd32f78" Oct 30 19:22:12.495968 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:22:12.495952 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0f29a229fd44421812be3865231c2954a10a8d17e421382155f16aa9bd32f78\": container with ID starting with d0f29a229fd44421812be3865231c2954a10a8d17e421382155f16aa9bd32f78 not found: ID does not exist" containerID="d0f29a229fd44421812be3865231c2954a10a8d17e421382155f16aa9bd32f78" Oct 30 19:22:12.496019 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:12.495976 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0f29a229fd44421812be3865231c2954a10a8d17e421382155f16aa9bd32f78"} err="failed to get container status \"d0f29a229fd44421812be3865231c2954a10a8d17e421382155f16aa9bd32f78\": rpc error: code = NotFound desc = could not find container \"d0f29a229fd44421812be3865231c2954a10a8d17e421382155f16aa9bd32f78\": container with ID starting with d0f29a229fd44421812be3865231c2954a10a8d17e421382155f16aa9bd32f78 not found: ID does not exist" Oct 30 19:22:12.496019 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:12.495991 5894 scope.go:122] "RemoveContainer" containerID="cdc0d40f50f26e4e40419f58a5a929522a0e258cae6deefd5c7127e887a48cc7" Oct 30 19:22:12.496205 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:22:12.496191 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdc0d40f50f26e4e40419f58a5a929522a0e258cae6deefd5c7127e887a48cc7\": container with ID starting with cdc0d40f50f26e4e40419f58a5a929522a0e258cae6deefd5c7127e887a48cc7 not found: ID does not exist" containerID="cdc0d40f50f26e4e40419f58a5a929522a0e258cae6deefd5c7127e887a48cc7" Oct 30 19:22:12.496245 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:12.496209 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdc0d40f50f26e4e40419f58a5a929522a0e258cae6deefd5c7127e887a48cc7"} err="failed to get container status \"cdc0d40f50f26e4e40419f58a5a929522a0e258cae6deefd5c7127e887a48cc7\": rpc error: code = NotFound desc = could not find container \"cdc0d40f50f26e4e40419f58a5a929522a0e258cae6deefd5c7127e887a48cc7\": container with ID starting with cdc0d40f50f26e4e40419f58a5a929522a0e258cae6deefd5c7127e887a48cc7 not found: ID does not exist" Oct 30 19:22:12.642252 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:12.642196 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4"] Oct 30 19:22:12.691942 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:12.691913 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-e0fc1f-openshift-default-7b5f88b885-fgsf4"] Oct 30 19:22:13.648620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:13.648568 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="b0479234-1db6-465f-9454-54d777b09538" path="/var/lib/kubelet/pods/b0479234-1db6-465f-9454-54d777b09538/volumes" Oct 30 19:22:16.354620 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:16.354394 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:17.628805 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:17.628759 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:17.695270 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:17.695236 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" podUID="eb9bc7c3-4527-4569-b896-b302d1bc5afc" containerName="payload-pre-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:22:18.702073 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:18.702038 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" podUID="1b395010-277d-42b3-b191-0e24f69156c1" containerName="payload-processing" probeResult="failure" output="service unhealthy (responded with \"NOT_SERVING\")" Oct 30 19:22:19.948875 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:19.948799 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:20.119440 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:22:20.119404 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:22:20.126399 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.126375 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:20.277307 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.277285 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" Oct 30 19:22:20.356945 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.356905 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1b395010-277d-42b3-b191-0e24f69156c1-kube-api-access-kxth6\" (UniqueName: \"kubernetes.io/projected/1b395010-277d-42b3-b191-0e24f69156c1-kube-api-access-kxth6\") pod \"1b395010-277d-42b3-b191-0e24f69156c1\" (UID: \"1b395010-277d-42b3-b191-0e24f69156c1\") " Oct 30 19:22:20.357122 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.356953 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1b395010-277d-42b3-b191-0e24f69156c1-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b395010-277d-42b3-b191-0e24f69156c1-plugins-config-volume\") pod \"1b395010-277d-42b3-b191-0e24f69156c1\" (UID: \"1b395010-277d-42b3-b191-0e24f69156c1\") " Oct 30 19:22:20.357389 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.357347 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b395010-277d-42b3-b191-0e24f69156c1-plugins-config-volume" pod "1b395010-277d-42b3-b191-0e24f69156c1" (UID: "1b395010-277d-42b3-b191-0e24f69156c1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:22:20.359613 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.359589 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b395010-277d-42b3-b191-0e24f69156c1-kube-api-access-kxth6" pod "1b395010-277d-42b3-b191-0e24f69156c1" (UID: "1b395010-277d-42b3-b191-0e24f69156c1"). InnerVolumeSpecName "kube-api-access-kxth6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:22:20.379243 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.379221 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" Oct 30 19:22:20.458161 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.458069 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/eb9bc7c3-4527-4569-b896-b302d1bc5afc-plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb9bc7c3-4527-4569-b896-b302d1bc5afc-plugins-config-volume\") pod \"eb9bc7c3-4527-4569-b896-b302d1bc5afc\" (UID: \"eb9bc7c3-4527-4569-b896-b302d1bc5afc\") " Oct 30 19:22:20.458333 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.458211 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/eb9bc7c3-4527-4569-b896-b302d1bc5afc-kube-api-access-cr2c4\" (UniqueName: \"kubernetes.io/projected/eb9bc7c3-4527-4569-b896-b302d1bc5afc-kube-api-access-cr2c4\") pod \"eb9bc7c3-4527-4569-b896-b302d1bc5afc\" (UID: \"eb9bc7c3-4527-4569-b896-b302d1bc5afc\") " Oct 30 19:22:20.458494 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.458463 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb9bc7c3-4527-4569-b896-b302d1bc5afc-plugins-config-volume" pod "eb9bc7c3-4527-4569-b896-b302d1bc5afc" (UID: "eb9bc7c3-4527-4569-b896-b302d1bc5afc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:22:20.458566 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.458552 5894 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1b395010-277d-42b3-b191-0e24f69156c1-plugins-config-volume\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:22:20.458648 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.458602 5894 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb9bc7c3-4527-4569-b896-b302d1bc5afc-plugins-config-volume\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:22:20.458648 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.458620 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kxth6\" (UniqueName: \"kubernetes.io/projected/1b395010-277d-42b3-b191-0e24f69156c1-kube-api-access-kxth6\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:22:20.460197 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.460178 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb9bc7c3-4527-4569-b896-b302d1bc5afc-kube-api-access-cr2c4" pod "eb9bc7c3-4527-4569-b896-b302d1bc5afc" (UID: "eb9bc7c3-4527-4569-b896-b302d1bc5afc"). InnerVolumeSpecName "kube-api-access-cr2c4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:22:20.523517 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.523476 5894 generic.go:356] "Generic (PLEG): container finished" podID="1b395010-277d-42b3-b191-0e24f69156c1" containerID="aa8be6fce6695580a0d73c25241124a922c7b7bb4b70f61e9796104c2b4b4d95" exitCode=0 Oct 30 19:22:20.523698 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.523537 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" Oct 30 19:22:20.523698 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.523538 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" event={"ID":"1b395010-277d-42b3-b191-0e24f69156c1","Type":"ContainerDied","Data":"aa8be6fce6695580a0d73c25241124a922c7b7bb4b70f61e9796104c2b4b4d95"} Oct 30 19:22:20.523698 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.523624 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct" event={"ID":"1b395010-277d-42b3-b191-0e24f69156c1","Type":"ContainerDied","Data":"8b2b2d2d18b350fe5e877eb58bd17ca9bbce292b126c945f1d8348553668afbc"} Oct 30 19:22:20.523698 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.523686 5894 scope.go:122] "RemoveContainer" containerID="aa8be6fce6695580a0d73c25241124a922c7b7bb4b70f61e9796104c2b4b4d95" Oct 30 19:22:20.525341 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.525314 5894 generic.go:356] "Generic (PLEG): container finished" podID="eb9bc7c3-4527-4569-b896-b302d1bc5afc" containerID="898b42ca95974646e1b62dbe90c030f7d79e238ab17ddffb3ccf42171e6f6cf5" exitCode=0 Oct 30 19:22:20.525500 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.525482 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" event={"ID":"eb9bc7c3-4527-4569-b896-b302d1bc5afc","Type":"ContainerDied","Data":"898b42ca95974646e1b62dbe90c030f7d79e238ab17ddffb3ccf42171e6f6cf5"} Oct 30 19:22:20.525563 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.525528 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" event={"ID":"eb9bc7c3-4527-4569-b896-b302d1bc5afc","Type":"ContainerDied","Data":"6e93e46a266f14a6d220a3a0db4a0ebebc2bca8751b0e02587e1b78b4396c3ca"} Oct 30 19:22:20.525638 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.525623 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs" Oct 30 19:22:20.534778 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.534693 5894 scope.go:122] "RemoveContainer" containerID="aa8be6fce6695580a0d73c25241124a922c7b7bb4b70f61e9796104c2b4b4d95" Oct 30 19:22:20.535050 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:22:20.535004 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa8be6fce6695580a0d73c25241124a922c7b7bb4b70f61e9796104c2b4b4d95\": container with ID starting with aa8be6fce6695580a0d73c25241124a922c7b7bb4b70f61e9796104c2b4b4d95 not found: ID does not exist" containerID="aa8be6fce6695580a0d73c25241124a922c7b7bb4b70f61e9796104c2b4b4d95" Oct 30 19:22:20.535126 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.535038 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa8be6fce6695580a0d73c25241124a922c7b7bb4b70f61e9796104c2b4b4d95"} err="failed to get container status \"aa8be6fce6695580a0d73c25241124a922c7b7bb4b70f61e9796104c2b4b4d95\": rpc error: code = NotFound desc = could not find container \"aa8be6fce6695580a0d73c25241124a922c7b7bb4b70f61e9796104c2b4b4d95\": container with ID starting with aa8be6fce6695580a0d73c25241124a922c7b7bb4b70f61e9796104c2b4b4d95 not found: ID does not exist" Oct 30 19:22:20.535126 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.535062 5894 scope.go:122] "RemoveContainer" containerID="898b42ca95974646e1b62dbe90c030f7d79e238ab17ddffb3ccf42171e6f6cf5" Oct 30 19:22:20.545261 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.545242 5894 scope.go:122] "RemoveContainer" containerID="898b42ca95974646e1b62dbe90c030f7d79e238ab17ddffb3ccf42171e6f6cf5" Oct 30 19:22:20.545593 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:22:20.545551 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"898b42ca95974646e1b62dbe90c030f7d79e238ab17ddffb3ccf42171e6f6cf5\": container with ID starting with 898b42ca95974646e1b62dbe90c030f7d79e238ab17ddffb3ccf42171e6f6cf5 not found: ID does not exist" containerID="898b42ca95974646e1b62dbe90c030f7d79e238ab17ddffb3ccf42171e6f6cf5" Oct 30 19:22:20.545685 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.545602 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"898b42ca95974646e1b62dbe90c030f7d79e238ab17ddffb3ccf42171e6f6cf5"} err="failed to get container status \"898b42ca95974646e1b62dbe90c030f7d79e238ab17ddffb3ccf42171e6f6cf5\": rpc error: code = NotFound desc = could not find container \"898b42ca95974646e1b62dbe90c030f7d79e238ab17ddffb3ccf42171e6f6cf5\": container with ID starting with 898b42ca95974646e1b62dbe90c030f7d79e238ab17ddffb3ccf42171e6f6cf5 not found: ID does not exist" Oct 30 19:22:20.560116 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.560093 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cr2c4\" (UniqueName: \"kubernetes.io/projected/eb9bc7c3-4527-4569-b896-b302d1bc5afc-kube-api-access-cr2c4\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:22:20.690727 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.690689 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct"] Oct 30 19:22:20.697404 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.697376 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e0fc1f-6d7bc77bc-244ct"] Oct 30 19:22:20.806435 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.806365 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs"] Oct 30 19:22:20.810171 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:20.810145 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e0fc1f-68f84c4755-w84bs"] Oct 30 19:22:21.209074 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:21.209040 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:21.648517 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:21.648487 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1b395010-277d-42b3-b191-0e24f69156c1" path="/var/lib/kubelet/pods/1b395010-277d-42b3-b191-0e24f69156c1/volumes" Oct 30 19:22:21.648935 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:21.648919 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="eb9bc7c3-4527-4569-b896-b302d1bc5afc" path="/var/lib/kubelet/pods/eb9bc7c3-4527-4569-b896-b302d1bc5afc/volumes" Oct 30 19:22:25.313213 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:25.313181 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:26.813053 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:26.813016 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:26.993953 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:26.993902 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:27.174729 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:27.174694 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:28.764299 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:28.764268 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:32.059827 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:32.059731 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:34.692345 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:34.692309 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:34.870148 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:34.870104 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:35.053568 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:35.053485 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:35.546439 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:35.546405 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:36.787198 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:36.787165 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:40.475545 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:40.475511 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l"] Oct 30 19:22:40.612959 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:40.612930 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:41.963553 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:41.963510 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:46.686858 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:46.686816 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:47.601021 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:47.600976 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:47.785872 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:47.785841 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:22:50.117464 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:22:50.117430 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:22:55.495586 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.495533 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" containerID="cri-o://dd3c4e0e6ca81dc3c99a39612bdaf93ee9d05b2253e7879cdcc22d3a2a598e0a" gracePeriod=45 Oct 30 19:22:55.701270 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.701238 5894 generic.go:356] "Generic (PLEG): container finished" podID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerID="dd3c4e0e6ca81dc3c99a39612bdaf93ee9d05b2253e7879cdcc22d3a2a598e0a" exitCode=0 Oct 30 19:22:55.701451 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.701304 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" event={"ID":"3c2df19e-78b6-4db7-bc32-db2bef552a2a","Type":"ContainerDied","Data":"dd3c4e0e6ca81dc3c99a39612bdaf93ee9d05b2253e7879cdcc22d3a2a598e0a"} Oct 30 19:22:55.746419 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.746363 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 19:22:55.793717 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.793692 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-tmp-dir\") pod \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " Oct 30 19:22:55.793852 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.793754 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/3c2df19e-78b6-4db7-bc32-db2bef552a2a-tls-certs\" (UniqueName: \"kubernetes.io/secret/3c2df19e-78b6-4db7-bc32-db2bef552a2a-tls-certs\") pod \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " Oct 30 19:22:55.793852 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.793795 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-model-cache\") pod \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " Oct 30 19:22:55.793852 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.793838 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-dshm\") pod \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " Oct 30 19:22:55.793969 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.793872 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3c2df19e-78b6-4db7-bc32-db2bef552a2a-kube-api-access-gv84j\" (UniqueName: \"kubernetes.io/projected/3c2df19e-78b6-4db7-bc32-db2bef552a2a-kube-api-access-gv84j\") pod \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " Oct 30 19:22:55.793969 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.793921 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-home\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-home\") pod \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\" (UID: \"3c2df19e-78b6-4db7-bc32-db2bef552a2a\") " Oct 30 19:22:55.794079 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.793956 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-tmp-dir" pod "3c2df19e-78b6-4db7-bc32-db2bef552a2a" (UID: "3c2df19e-78b6-4db7-bc32-db2bef552a2a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:22:55.794079 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.794065 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-model-cache" pod "3c2df19e-78b6-4db7-bc32-db2bef552a2a" (UID: "3c2df19e-78b6-4db7-bc32-db2bef552a2a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:22:55.794259 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.794230 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-home" pod "3c2df19e-78b6-4db7-bc32-db2bef552a2a" (UID: "3c2df19e-78b6-4db7-bc32-db2bef552a2a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:22:55.794332 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.794264 5894 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-tmp-dir\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:22:55.794332 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.794280 5894 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-model-cache\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:22:55.796116 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.796092 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3c2df19e-78b6-4db7-bc32-db2bef552a2a-kube-api-access-gv84j" pod "3c2df19e-78b6-4db7-bc32-db2bef552a2a" (UID: "3c2df19e-78b6-4db7-bc32-db2bef552a2a"). InnerVolumeSpecName "kube-api-access-gv84j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:22:55.796116 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.796108 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3c2df19e-78b6-4db7-bc32-db2bef552a2a-tls-certs" pod "3c2df19e-78b6-4db7-bc32-db2bef552a2a" (UID: "3c2df19e-78b6-4db7-bc32-db2bef552a2a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:22:55.796248 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.796124 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-dshm" pod "3c2df19e-78b6-4db7-bc32-db2bef552a2a" (UID: "3c2df19e-78b6-4db7-bc32-db2bef552a2a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:22:55.895495 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.895464 5894 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3c2df19e-78b6-4db7-bc32-db2bef552a2a-tls-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:22:55.895495 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.895490 5894 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-dshm\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:22:55.895495 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.895500 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gv84j\" (UniqueName: \"kubernetes.io/projected/3c2df19e-78b6-4db7-bc32-db2bef552a2a-kube-api-access-gv84j\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:22:55.895743 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:55.895510 5894 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3c2df19e-78b6-4db7-bc32-db2bef552a2a-home\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:22:56.707504 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:56.707469 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" Oct 30 19:22:56.708057 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:56.707499 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l" event={"ID":"3c2df19e-78b6-4db7-bc32-db2bef552a2a","Type":"ContainerDied","Data":"b019dabcdb46dab9f1babb8a6983201e6ea0794820ef08500c0a7932a4ca5d56"} Oct 30 19:22:56.708057 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:56.707552 5894 scope.go:122] "RemoveContainer" containerID="dd3c4e0e6ca81dc3c99a39612bdaf93ee9d05b2253e7879cdcc22d3a2a598e0a" Oct 30 19:22:56.871871 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:56.871826 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l"] Oct 30 19:22:56.921547 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:56.921518 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-66bdc0/premium-w2-66bdc0-kserve-f98cf4dcf-h957l"] Oct 30 19:22:57.649425 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:22:57.649389 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" path="/var/lib/kubelet/pods/3c2df19e-78b6-4db7-bc32-db2bef552a2a/volumes" Oct 30 19:23:06.106228 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.106193 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699"] Oct 30 19:23:06.106772 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.106382 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" podUID="d7439fb6-eee6-483a-b43b-1f5bef9b1d49" containerName="maas-api" containerID="cri-o://2a9a1debbdbf0f4fcc7f3c7051ee51a5a84a30a0f1baed0f5a21d172b2e4d6e6" gracePeriod=30 Oct 30 19:23:06.264229 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.264195 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:06.379849 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.379823 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" Oct 30 19:23:06.499176 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.499134 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-kube-api-access-mcgtf\" (UniqueName: \"kubernetes.io/projected/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-kube-api-access-mcgtf\") pod \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\" (UID: \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\") " Oct 30 19:23:06.499375 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.499203 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-maas-api-tls\") pod \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\" (UID: \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\") " Oct 30 19:23:06.499375 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.499246 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-openshift-service-ca\") pod \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\" (UID: \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\") " Oct 30 19:23:06.499375 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.499317 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-metrics-certs\") pod \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\" (UID: \"d7439fb6-eee6-483a-b43b-1f5bef9b1d49\") " Oct 30 19:23:06.501068 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.501031 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-openshift-service-ca" pod "d7439fb6-eee6-483a-b43b-1f5bef9b1d49" (UID: "d7439fb6-eee6-483a-b43b-1f5bef9b1d49"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:23:06.509190 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.504878 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-kube-api-access-mcgtf" pod "d7439fb6-eee6-483a-b43b-1f5bef9b1d49" (UID: "d7439fb6-eee6-483a-b43b-1f5bef9b1d49"). InnerVolumeSpecName "kube-api-access-mcgtf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:23:06.509190 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.509108 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-maas-api-tls" pod "d7439fb6-eee6-483a-b43b-1f5bef9b1d49" (UID: "d7439fb6-eee6-483a-b43b-1f5bef9b1d49"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:23:06.513793 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.513556 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-metrics-certs" pod "d7439fb6-eee6-483a-b43b-1f5bef9b1d49" (UID: "d7439fb6-eee6-483a-b43b-1f5bef9b1d49"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:23:06.600352 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.600314 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mcgtf\" (UniqueName: \"kubernetes.io/projected/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-kube-api-access-mcgtf\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:23:06.600502 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.600366 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-maas-api-tls\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:23:06.600502 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.600384 5894 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-openshift-service-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:23:06.600502 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.600397 5894 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7439fb6-eee6-483a-b43b-1f5bef9b1d49-metrics-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:23:06.757014 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.756979 5894 generic.go:356] "Generic (PLEG): container finished" podID="d7439fb6-eee6-483a-b43b-1f5bef9b1d49" containerID="2a9a1debbdbf0f4fcc7f3c7051ee51a5a84a30a0f1baed0f5a21d172b2e4d6e6" exitCode=0 Oct 30 19:23:06.757218 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.757023 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" Oct 30 19:23:06.757218 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.757035 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" event={"ID":"d7439fb6-eee6-483a-b43b-1f5bef9b1d49","Type":"ContainerDied","Data":"2a9a1debbdbf0f4fcc7f3c7051ee51a5a84a30a0f1baed0f5a21d172b2e4d6e6"} Oct 30 19:23:06.757218 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.757088 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699" event={"ID":"d7439fb6-eee6-483a-b43b-1f5bef9b1d49","Type":"ContainerDied","Data":"eb2e4ae1a6f7d3183602aaaf09a8a67d5146964f648b53a769753dc7c3e92be4"} Oct 30 19:23:06.757218 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.757116 5894 scope.go:122] "RemoveContainer" containerID="2a9a1debbdbf0f4fcc7f3c7051ee51a5a84a30a0f1baed0f5a21d172b2e4d6e6" Oct 30 19:23:06.766684 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.766659 5894 scope.go:122] "RemoveContainer" containerID="2a9a1debbdbf0f4fcc7f3c7051ee51a5a84a30a0f1baed0f5a21d172b2e4d6e6" Oct 30 19:23:06.766953 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:23:06.766931 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a9a1debbdbf0f4fcc7f3c7051ee51a5a84a30a0f1baed0f5a21d172b2e4d6e6\": container with ID starting with 2a9a1debbdbf0f4fcc7f3c7051ee51a5a84a30a0f1baed0f5a21d172b2e4d6e6 not found: ID does not exist" containerID="2a9a1debbdbf0f4fcc7f3c7051ee51a5a84a30a0f1baed0f5a21d172b2e4d6e6" Oct 30 19:23:06.767016 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.766963 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a9a1debbdbf0f4fcc7f3c7051ee51a5a84a30a0f1baed0f5a21d172b2e4d6e6"} err="failed to get container status \"2a9a1debbdbf0f4fcc7f3c7051ee51a5a84a30a0f1baed0f5a21d172b2e4d6e6\": rpc error: code = NotFound desc = could not find container \"2a9a1debbdbf0f4fcc7f3c7051ee51a5a84a30a0f1baed0f5a21d172b2e4d6e6\": container with ID starting with 2a9a1debbdbf0f4fcc7f3c7051ee51a5a84a30a0f1baed0f5a21d172b2e4d6e6 not found: ID does not exist" Oct 30 19:23:06.918746 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.918716 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699"] Oct 30 19:23:06.920597 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:06.920559 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-66bdc0-dc868d9cf-9b699"] Oct 30 19:23:07.649596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:07.649536 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="d7439fb6-eee6-483a-b43b-1f5bef9b1d49" path="/var/lib/kubelet/pods/d7439fb6-eee6-483a-b43b-1f5bef9b1d49/volumes" Oct 30 19:23:07.840314 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:07.840277 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:08.044969 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:08.044847 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:08.715419 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:08.715388 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:15.353658 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:15.353622 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:15.560951 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:15.560914 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:15.654654 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:15.654625 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl"] Oct 30 19:23:15.654824 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:15.654782 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" podUID="9e867658-2d43-4edc-9d26-f23fdd765a80" containerName="istio-proxy" containerID="cri-o://29edc299346db07618b499a54e63bd4392058106256f2828ce551469ff899996" gracePeriod=30 Oct 30 19:23:15.738201 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:15.735976 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:17.305837 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:17.305804 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:17.509700 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:17.509662 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:18.590553 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:18.590474 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:20.116784 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:23:20.116749 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:23:20.838874 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:20.838842 5894 generic.go:356] "Generic (PLEG): container finished" podID="9e867658-2d43-4edc-9d26-f23fdd765a80" containerID="29edc299346db07618b499a54e63bd4392058106256f2828ce551469ff899996" exitCode=0 Oct 30 19:23:20.839037 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:20.838911 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" event={"ID":"9e867658-2d43-4edc-9d26-f23fdd765a80","Type":"ContainerDied","Data":"29edc299346db07618b499a54e63bd4392058106256f2828ce551469ff899996"} Oct 30 19:23:20.910037 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:20.910015 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 19:23:21.013172 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.013091 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-994975fdf-8p5jh"] Oct 30 19:23:21.013318 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.013270 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/maas-controller-994975fdf-8p5jh" podUID="5264ce31-dc5b-42ed-a76e-631aee315933" containerName="manager" containerID="cri-o://e71b6ad01758518a08d45087a2a36d510b62c9249007e44425a848c380610233" gracePeriod=30 Oct 30 19:23:21.044355 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.044330 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/9e867658-2d43-4edc-9d26-f23fdd765a80-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e867658-2d43-4edc-9d26-f23fdd765a80-istiod-ca-cert\") pod \"9e867658-2d43-4edc-9d26-f23fdd765a80\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " Oct 30 19:23:21.044454 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.044368 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-token\" (UniqueName: \"kubernetes.io/projected/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-token\") pod \"9e867658-2d43-4edc-9d26-f23fdd765a80\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " Oct 30 19:23:21.044454 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.044407 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-workload-certs\") pod \"9e867658-2d43-4edc-9d26-f23fdd765a80\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " Oct 30 19:23:21.044454 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.044439 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9e867658-2d43-4edc-9d26-f23fdd765a80-kube-api-access-67lp9\" (UniqueName: \"kubernetes.io/projected/9e867658-2d43-4edc-9d26-f23fdd765a80-kube-api-access-67lp9\") pod \"9e867658-2d43-4edc-9d26-f23fdd765a80\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " Oct 30 19:23:21.044643 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.044459 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-workload-socket\") pod \"9e867658-2d43-4edc-9d26-f23fdd765a80\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " Oct 30 19:23:21.044643 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.044491 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-data\") pod \"9e867658-2d43-4edc-9d26-f23fdd765a80\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " Oct 30 19:23:21.044643 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.044547 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-credential-socket\") pod \"9e867658-2d43-4edc-9d26-f23fdd765a80\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " Oct 30 19:23:21.044643 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.044594 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-podinfo\") pod \"9e867658-2d43-4edc-9d26-f23fdd765a80\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " Oct 30 19:23:21.044863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.044655 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-envoy\") pod \"9e867658-2d43-4edc-9d26-f23fdd765a80\" (UID: \"9e867658-2d43-4edc-9d26-f23fdd765a80\") " Oct 30 19:23:21.044863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.044722 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9e867658-2d43-4edc-9d26-f23fdd765a80-istiod-ca-cert" pod "9e867658-2d43-4edc-9d26-f23fdd765a80" (UID: "9e867658-2d43-4edc-9d26-f23fdd765a80"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:23:21.044863 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.044815 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-workload-certs" pod "9e867658-2d43-4edc-9d26-f23fdd765a80" (UID: "9e867658-2d43-4edc-9d26-f23fdd765a80"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:23:21.045027 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.044930 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-credential-socket" pod "9e867658-2d43-4edc-9d26-f23fdd765a80" (UID: "9e867658-2d43-4edc-9d26-f23fdd765a80"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:23:21.045027 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.045012 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-data" pod "9e867658-2d43-4edc-9d26-f23fdd765a80" (UID: "9e867658-2d43-4edc-9d26-f23fdd765a80"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:23:21.045146 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.045023 5894 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9e867658-2d43-4edc-9d26-f23fdd765a80-istiod-ca-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:23:21.045146 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.045045 5894 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-workload-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:23:21.045146 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.045051 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-workload-socket" pod "9e867658-2d43-4edc-9d26-f23fdd765a80" (UID: "9e867658-2d43-4edc-9d26-f23fdd765a80"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:23:21.045146 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.045059 5894 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-credential-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:23:21.046690 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.046669 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-token" pod "9e867658-2d43-4edc-9d26-f23fdd765a80" (UID: "9e867658-2d43-4edc-9d26-f23fdd765a80"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:23:21.046792 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.046724 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e867658-2d43-4edc-9d26-f23fdd765a80-kube-api-access-67lp9" pod "9e867658-2d43-4edc-9d26-f23fdd765a80" (UID: "9e867658-2d43-4edc-9d26-f23fdd765a80"). InnerVolumeSpecName "kube-api-access-67lp9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:23:21.046836 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.046805 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-envoy" pod "9e867658-2d43-4edc-9d26-f23fdd765a80" (UID: "9e867658-2d43-4edc-9d26-f23fdd765a80"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Oct 30 19:23:21.047396 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.047376 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-podinfo" pod "9e867658-2d43-4edc-9d26-f23fdd765a80" (UID: "9e867658-2d43-4edc-9d26-f23fdd765a80"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Oct 30 19:23:21.146175 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.146150 5894 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-envoy\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:23:21.146175 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.146174 5894 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-token\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:23:21.146549 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.146184 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-67lp9\" (UniqueName: \"kubernetes.io/projected/9e867658-2d43-4edc-9d26-f23fdd765a80-kube-api-access-67lp9\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:23:21.146549 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.146194 5894 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-workload-socket\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:23:21.146549 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.146203 5894 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-data\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:23:21.146549 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.146213 5894 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9e867658-2d43-4edc-9d26-f23fdd765a80-istio-podinfo\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:23:21.251899 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.251858 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-994975fdf-8p5jh" Oct 30 19:23:21.348443 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.348373 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5264ce31-dc5b-42ed-a76e-631aee315933-kube-api-access-f58pm\" (UniqueName: \"kubernetes.io/projected/5264ce31-dc5b-42ed-a76e-631aee315933-kube-api-access-f58pm\") pod \"5264ce31-dc5b-42ed-a76e-631aee315933\" (UID: \"5264ce31-dc5b-42ed-a76e-631aee315933\") " Oct 30 19:23:21.348443 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.348437 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5264ce31-dc5b-42ed-a76e-631aee315933-webhook-cert\" (UniqueName: \"kubernetes.io/secret/5264ce31-dc5b-42ed-a76e-631aee315933-webhook-cert\") pod \"5264ce31-dc5b-42ed-a76e-631aee315933\" (UID: \"5264ce31-dc5b-42ed-a76e-631aee315933\") " Oct 30 19:23:21.348708 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.348493 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5264ce31-dc5b-42ed-a76e-631aee315933-metrics-certs\" (UniqueName: \"kubernetes.io/secret/5264ce31-dc5b-42ed-a76e-631aee315933-metrics-certs\") pod \"5264ce31-dc5b-42ed-a76e-631aee315933\" (UID: \"5264ce31-dc5b-42ed-a76e-631aee315933\") " Oct 30 19:23:21.350462 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.350432 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5264ce31-dc5b-42ed-a76e-631aee315933-webhook-cert" pod "5264ce31-dc5b-42ed-a76e-631aee315933" (UID: "5264ce31-dc5b-42ed-a76e-631aee315933"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:23:21.350595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.350464 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5264ce31-dc5b-42ed-a76e-631aee315933-metrics-certs" pod "5264ce31-dc5b-42ed-a76e-631aee315933" (UID: "5264ce31-dc5b-42ed-a76e-631aee315933"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:23:21.350595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.350488 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5264ce31-dc5b-42ed-a76e-631aee315933-kube-api-access-f58pm" pod "5264ce31-dc5b-42ed-a76e-631aee315933" (UID: "5264ce31-dc5b-42ed-a76e-631aee315933"). InnerVolumeSpecName "kube-api-access-f58pm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:23:21.450082 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.450052 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f58pm\" (UniqueName: \"kubernetes.io/projected/5264ce31-dc5b-42ed-a76e-631aee315933-kube-api-access-f58pm\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:23:21.450082 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.450080 5894 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5264ce31-dc5b-42ed-a76e-631aee315933-webhook-cert\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:23:21.450082 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.450089 5894 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5264ce31-dc5b-42ed-a76e-631aee315933-metrics-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:23:21.845399 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.845368 5894 generic.go:356] "Generic (PLEG): container finished" podID="5264ce31-dc5b-42ed-a76e-631aee315933" containerID="e71b6ad01758518a08d45087a2a36d510b62c9249007e44425a848c380610233" exitCode=0 Oct 30 19:23:21.845557 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.845409 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-994975fdf-8p5jh" Oct 30 19:23:21.845557 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.845450 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-994975fdf-8p5jh" event={"ID":"5264ce31-dc5b-42ed-a76e-631aee315933","Type":"ContainerDied","Data":"e71b6ad01758518a08d45087a2a36d510b62c9249007e44425a848c380610233"} Oct 30 19:23:21.845557 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.845506 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-994975fdf-8p5jh" event={"ID":"5264ce31-dc5b-42ed-a76e-631aee315933","Type":"ContainerDied","Data":"f2ea90a2e452b42353eba57811bd8ac5a6575e32d5a47a9cd25958aa54a77e88"} Oct 30 19:23:21.845557 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.845523 5894 scope.go:122] "RemoveContainer" containerID="e71b6ad01758518a08d45087a2a36d510b62c9249007e44425a848c380610233" Oct 30 19:23:21.846820 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.846797 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" event={"ID":"9e867658-2d43-4edc-9d26-f23fdd765a80","Type":"ContainerDied","Data":"7221e1b8d3a5766a115503b17f969cf2c1dab570f93da9ba8bfbb7fa0d4f9084"} Oct 30 19:23:21.846947 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.846842 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl" Oct 30 19:23:21.855405 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.855386 5894 scope.go:122] "RemoveContainer" containerID="e71b6ad01758518a08d45087a2a36d510b62c9249007e44425a848c380610233" Oct 30 19:23:21.855699 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:23:21.855681 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e71b6ad01758518a08d45087a2a36d510b62c9249007e44425a848c380610233\": container with ID starting with e71b6ad01758518a08d45087a2a36d510b62c9249007e44425a848c380610233 not found: ID does not exist" containerID="e71b6ad01758518a08d45087a2a36d510b62c9249007e44425a848c380610233" Oct 30 19:23:21.855750 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.855708 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e71b6ad01758518a08d45087a2a36d510b62c9249007e44425a848c380610233"} err="failed to get container status \"e71b6ad01758518a08d45087a2a36d510b62c9249007e44425a848c380610233\": rpc error: code = NotFound desc = could not find container \"e71b6ad01758518a08d45087a2a36d510b62c9249007e44425a848c380610233\": container with ID starting with e71b6ad01758518a08d45087a2a36d510b62c9249007e44425a848c380610233 not found: ID does not exist" Oct 30 19:23:21.855750 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:21.855726 5894 scope.go:122] "RemoveContainer" containerID="29edc299346db07618b499a54e63bd4392058106256f2828ce551469ff899996" Oct 30 19:23:22.008832 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:22.008754 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-994975fdf-8p5jh"] Oct 30 19:23:22.058351 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:22.058320 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-994975fdf-8p5jh"] Oct 30 19:23:22.170851 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:22.170811 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl"] Oct 30 19:23:22.172166 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:22.172147 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-66bdc0-openshift-default-7db494c579-cjnsl"] Oct 30 19:23:23.650082 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:23.650036 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5264ce31-dc5b-42ed-a76e-631aee315933" path="/var/lib/kubelet/pods/5264ce31-dc5b-42ed-a76e-631aee315933/volumes" Oct 30 19:23:23.650728 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:23.650703 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9e867658-2d43-4edc-9d26-f23fdd765a80" path="/var/lib/kubelet/pods/9e867658-2d43-4edc-9d26-f23fdd765a80/volumes" Oct 30 19:23:24.114140 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:24.114060 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:27.144353 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:27.144313 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:23:50.123901 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:23:50.123852 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:23:50.304691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:50.304655 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 19:23:50.327110 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:50.327076 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 19:23:58.900137 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:23:58.900100 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:24:01.103407 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:24:01.103325 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:24:20.116593 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:24:20.116545 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:24:27.569501 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:24:27.569468 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:24:50.117059 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:24:50.117019 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:25:20.116280 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:25:20.116243 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:25:27.836150 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:25:27.836115 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:25:50.117443 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:25:50.117365 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:26:20.116760 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:26:20.116724 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:26:28.124955 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:26:28.124918 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:26:50.116700 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:26:50.116660 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:26:57.169730 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:26:57.169691 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:26:57.359297 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:26:57.359263 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:03.087042 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:27:03.086966 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:20.117001 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:27:20.116964 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:27:28.508144 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:27:28.508109 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:41.070035 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:27:41.070000 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:27:50.116435 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:27:50.116397 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:28:03.559598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:28:03.559139 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-7z7fl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:28:03.559598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:28:03.559226 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" podUID="75f81dfd-86e7-4532-a20c-42423f908903" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:28:03.560697 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:28:03.560664 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-nvv4s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:28:03.560937 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:28:03.560908 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" podUID="3fec8f46-c372-49cc-8c0a-9cb885f465be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:28:03.602731 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:28:03.602687 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-w22dg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:28:03.603027 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:28:03.602994 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" podUID="010f73a2-9a20-49a6-8d88-079a27465e78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:28:20.136304 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:28:20.136255 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:28:28.850546 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:28:28.850511 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:28:50.116619 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:28:50.116514 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:28:50.335676 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:28:50.335648 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 19:28:50.359214 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:28:50.359189 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 19:29:04.756202 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:04.756162 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:14.272864 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:14.272824 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:14.874652 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:14.874614 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:15.187457 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:15.187421 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:19.448560 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:19.448518 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:19.777674 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:19.777597 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:19.957972 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:19.957930 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:20.116309 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:29:20.116225 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:29:22.658679 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:22.658641 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:22.834789 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:22.834749 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:25.674490 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:25.674449 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:25.848057 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:25.848017 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:26.030127 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:26.030030 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:29.256831 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:29.256794 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:30.500931 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:30.500889 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:35.396117 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:35.396077 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:40.202278 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:40.202239 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:44.406448 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:44.406405 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:45.094010 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:45.093975 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:45.295414 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:45.295365 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:48.188009 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:48.187968 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:50.119718 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:50.119675 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:50.135898 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:29:50.135862 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:29:54.901033 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:54.900996 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:29:59.709919 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:29:59.709880 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:04.469133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:30:04.469051 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:09.069058 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:30:09.069022 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:13.914137 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:30:13.914099 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:18.589514 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:30:18.589469 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:20.116914 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:30:20.116875 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:30:23.410178 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:30:23.410110 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:28.060813 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:30:28.060770 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:29.515979 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:30:29.515940 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:32.722235 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:30:32.722192 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:36.132942 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:30:36.132897 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:37.468111 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:30:37.468064 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:41.238708 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:30:41.238667 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:42.587767 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:30:42.587529 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:47.265835 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:30:47.265797 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:48.253903 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:30:48.253820 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:48.435681 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:30:48.435651 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:50.116733 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:30:50.116698 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:30:51.372711 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:30:51.372668 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:53.123940 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:30:53.123907 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:53.308004 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:30:53.307947 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:30:53.913062 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:30:53.913029 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:00.116011 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:31:00.115973 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:00.306247 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:31:00.306217 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:00.493744 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:31:00.493709 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:17.954677 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:31:17.954638 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:18.508318 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:31:18.508286 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:20.117006 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:31:20.116970 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:31:24.030833 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:31:24.030795 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:29.860488 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:31:29.860451 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:31:50.116909 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:31:50.116823 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:32:11.460554 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:32:11.460515 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:20.116779 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:32:20.116739 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:32:30.201325 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:32:30.201278 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:32:50.132498 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:32:50.132452 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:33:20.116589 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:33:20.116537 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:33:25.395595 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:25.395541 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:26.378288 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:26.378209 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:26.791448 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:26.791414 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:30.529347 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:30.529314 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:30.771474 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:30.771441 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:30.966391 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:30.966361 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:31.171043 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:31.171010 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:33.899132 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:33.899100 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:34.126426 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:34.126393 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:36.933077 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:36.933040 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:37.121646 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:37.121606 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:37.296356 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:37.296270 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:41.836550 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:41.836515 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:46.730381 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:46.730348 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:50.116614 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:33:50.116563 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:33:50.363847 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:50.363818 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 19:33:50.390802 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:50.390777 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 19:33:51.428756 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:51.428722 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:55.707704 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:55.707663 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:58.709923 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:58.709889 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:58.913020 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:58.912979 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:33:59.272532 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:33:59.272499 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:01.293532 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:01.293495 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:06.099825 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:06.099785 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:10.781736 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:10.781691 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:15.463201 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:15.463138 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:20.112538 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:20.112501 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:20.134889 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:34:20.134853 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:34:24.804943 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:24.804898 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:29.662416 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:29.662376 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:30.899679 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:30.899623 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:34.357725 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:34.357691 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:39.117123 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:39.117061 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:40.605199 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:40.605155 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:44.260912 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:44.260868 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:45.644279 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:45.644241 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:49.042147 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:49.042108 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:50.116814 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:34:50.116771 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:34:52.674411 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:52.674376 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:52.862543 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:52.862509 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:53.149905 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:53.149846 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:55.627719 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:55.627679 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:55.807550 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:55.807519 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:56.809251 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:56.809215 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:57.086917 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:57.086829 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:34:57.824011 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:34:57.823980 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:35:00.930915 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:35:00.930876 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:35:03.656629 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:35:03.656560 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:35:03.865666 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:35:03.865633 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:35:04.063836 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:35:04.063739 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:35:11.355583 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:35:11.355520 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:35:18.566598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:35:18.551677 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-7z7fl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.127:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:35:18.566598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:35:18.551746 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-7z7fl" podUID="75f81dfd-86e7-4532-a20c-42423f908903" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.127:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:35:18.566598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:35:18.562658 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-nvv4s container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.128:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:35:18.566598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:35:18.562745 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-nvv4s" podUID="3fec8f46-c372-49cc-8c0a-9cb885f465be" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.128:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:35:18.593781 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:35:18.593730 5894 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-6db586dd6c-w22dg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.126:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Oct 30 19:35:18.594057 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:35:18.594020 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-6db586dd6c-w22dg" podUID="010f73a2-9a20-49a6-8d88-079a27465e78" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.126:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Oct 30 19:35:20.116543 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:35:20.116496 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:35:21.762947 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:35:21.762908 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:35:22.943492 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:35:22.943440 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:35:31.217828 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:35:31.217789 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:35:31.880526 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:35:31.880491 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:35:37.339329 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:35:37.339296 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:35:50.116234 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:35:50.116199 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:35:54.564445 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:35:54.564407 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:35:54.747426 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:35:54.747387 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:35:54.923223 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:35:54.923190 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:36:20.125268 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:36:20.125226 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:36:30.473598 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:36:30.473542 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:36:31.538994 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:36:31.538958 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:36:45.645935 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:36:45.645901 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:36:50.124687 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:36:50.124630 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:37:20.116274 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:37:20.116233 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:37:31.895434 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:31.895364 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:37:50.117212 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:37:50.117172 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:37:50.321492 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:50.321458 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-msjhg"] Oct 30 19:37:50.321696 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:50.321671 5894 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" podUID="49d04e0a-032d-4491-a4ea-be136c886f8a" containerName="maas-api" containerID="cri-o://b9dfea8501499f8cba4e17479118518ba4c11a0c884b00e6ebf6b4b78d1ca1dd" gracePeriod=30 Oct 30 19:37:51.301154 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:51.301115 5894 prober.go:122] "Probe failed" probeType="Readiness" pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" podUID="49d04e0a-032d-4491-a4ea-be136c886f8a" containerName="maas-api" probeResult="failure" output="Get \"https://10.132.0.45:8443/health\": dial tcp 10.132.0.45:8443: connect: connection refused" Oct 30 19:37:51.578476 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:51.578452 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" Oct 30 19:37:51.663425 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:51.663396 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/49d04e0a-032d-4491-a4ea-be136c886f8a-kube-api-access-rnvx5\" (UniqueName: \"kubernetes.io/projected/49d04e0a-032d-4491-a4ea-be136c886f8a-kube-api-access-rnvx5\") pod \"49d04e0a-032d-4491-a4ea-be136c886f8a\" (UID: \"49d04e0a-032d-4491-a4ea-be136c886f8a\") " Oct 30 19:37:51.663566 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:51.663445 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/49d04e0a-032d-4491-a4ea-be136c886f8a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d04e0a-032d-4491-a4ea-be136c886f8a-openshift-service-ca\") pod \"49d04e0a-032d-4491-a4ea-be136c886f8a\" (UID: \"49d04e0a-032d-4491-a4ea-be136c886f8a\") " Oct 30 19:37:51.663566 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:51.663472 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/49d04e0a-032d-4491-a4ea-be136c886f8a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/49d04e0a-032d-4491-a4ea-be136c886f8a-metrics-certs\") pod \"49d04e0a-032d-4491-a4ea-be136c886f8a\" (UID: \"49d04e0a-032d-4491-a4ea-be136c886f8a\") " Oct 30 19:37:51.663691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:51.663569 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/49d04e0a-032d-4491-a4ea-be136c886f8a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d04e0a-032d-4491-a4ea-be136c886f8a-maas-api-tls\") pod \"49d04e0a-032d-4491-a4ea-be136c886f8a\" (UID: \"49d04e0a-032d-4491-a4ea-be136c886f8a\") " Oct 30 19:37:51.663847 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:51.663817 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49d04e0a-032d-4491-a4ea-be136c886f8a-openshift-service-ca" pod "49d04e0a-032d-4491-a4ea-be136c886f8a" (UID: "49d04e0a-032d-4491-a4ea-be136c886f8a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:37:51.664066 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:51.663925 5894 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/49d04e0a-032d-4491-a4ea-be136c886f8a-openshift-service-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:37:51.665610 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:51.665569 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49d04e0a-032d-4491-a4ea-be136c886f8a-metrics-certs" pod "49d04e0a-032d-4491-a4ea-be136c886f8a" (UID: "49d04e0a-032d-4491-a4ea-be136c886f8a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:37:51.665740 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:51.665695 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/49d04e0a-032d-4491-a4ea-be136c886f8a-kube-api-access-rnvx5" pod "49d04e0a-032d-4491-a4ea-be136c886f8a" (UID: "49d04e0a-032d-4491-a4ea-be136c886f8a"). InnerVolumeSpecName "kube-api-access-rnvx5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:37:51.665740 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:51.665722 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49d04e0a-032d-4491-a4ea-be136c886f8a-maas-api-tls" pod "49d04e0a-032d-4491-a4ea-be136c886f8a" (UID: "49d04e0a-032d-4491-a4ea-be136c886f8a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:37:51.765173 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:51.765143 5894 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/49d04e0a-032d-4491-a4ea-be136c886f8a-metrics-certs\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:37:51.765173 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:51.765168 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/49d04e0a-032d-4491-a4ea-be136c886f8a-maas-api-tls\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:37:51.765173 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:51.765180 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rnvx5\" (UniqueName: \"kubernetes.io/projected/49d04e0a-032d-4491-a4ea-be136c886f8a-kube-api-access-rnvx5\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:37:52.223379 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:52.223345 5894 generic.go:356] "Generic (PLEG): container finished" podID="49d04e0a-032d-4491-a4ea-be136c886f8a" containerID="b9dfea8501499f8cba4e17479118518ba4c11a0c884b00e6ebf6b4b78d1ca1dd" exitCode=0 Oct 30 19:37:52.223604 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:52.223394 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" Oct 30 19:37:52.223604 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:52.223428 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" event={"ID":"49d04e0a-032d-4491-a4ea-be136c886f8a","Type":"ContainerDied","Data":"b9dfea8501499f8cba4e17479118518ba4c11a0c884b00e6ebf6b4b78d1ca1dd"} Oct 30 19:37:52.223604 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:52.223490 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5c48877c75-msjhg" event={"ID":"49d04e0a-032d-4491-a4ea-be136c886f8a","Type":"ContainerDied","Data":"badd9d1904f714eb5732058c0bca0736921024100b9ba9123eafa10778c2bd91"} Oct 30 19:37:52.223604 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:52.223521 5894 scope.go:122] "RemoveContainer" containerID="b9dfea8501499f8cba4e17479118518ba4c11a0c884b00e6ebf6b4b78d1ca1dd" Oct 30 19:37:52.233080 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:52.233058 5894 scope.go:122] "RemoveContainer" containerID="b9dfea8501499f8cba4e17479118518ba4c11a0c884b00e6ebf6b4b78d1ca1dd" Oct 30 19:37:52.233365 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:37:52.233346 5894 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b9dfea8501499f8cba4e17479118518ba4c11a0c884b00e6ebf6b4b78d1ca1dd\": container with ID starting with b9dfea8501499f8cba4e17479118518ba4c11a0c884b00e6ebf6b4b78d1ca1dd not found: ID does not exist" containerID="b9dfea8501499f8cba4e17479118518ba4c11a0c884b00e6ebf6b4b78d1ca1dd" Oct 30 19:37:52.233436 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:52.233374 5894 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b9dfea8501499f8cba4e17479118518ba4c11a0c884b00e6ebf6b4b78d1ca1dd"} err="failed to get container status \"b9dfea8501499f8cba4e17479118518ba4c11a0c884b00e6ebf6b4b78d1ca1dd\": rpc error: code = NotFound desc = could not find container \"b9dfea8501499f8cba4e17479118518ba4c11a0c884b00e6ebf6b4b78d1ca1dd\": container with ID starting with b9dfea8501499f8cba4e17479118518ba4c11a0c884b00e6ebf6b4b78d1ca1dd not found: ID does not exist" Oct 30 19:37:52.386826 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:52.386796 5894 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-msjhg"] Oct 30 19:37:52.436710 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:52.436686 5894 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-5c48877c75-msjhg"] Oct 30 19:37:52.774488 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:52.774452 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:37:53.648928 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:37:53.648897 5894 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="49d04e0a-032d-4491-a4ea-be136c886f8a" path="/var/lib/kubelet/pods/49d04e0a-032d-4491-a4ea-be136c886f8a/volumes" Oct 30 19:38:02.833644 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.833605 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq"] Oct 30 19:38:02.834188 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834167 5894 cpu_manager.go:414] "containerMap: removing container" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" Oct 30 19:38:02.834275 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834191 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" Oct 30 19:38:02.834275 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834218 5894 cpu_manager.go:414] "containerMap: removing container" podUID="8db11496-7f7c-49f1-ba40-08248a5b2db2" containerName="istio-proxy" Oct 30 19:38:02.834275 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834228 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8db11496-7f7c-49f1-ba40-08248a5b2db2" containerName="istio-proxy" Oct 30 19:38:02.834275 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834244 5894 cpu_manager.go:414] "containerMap: removing container" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" Oct 30 19:38:02.834275 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834252 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" Oct 30 19:38:02.834275 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834263 5894 cpu_manager.go:414] "containerMap: removing container" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" Oct 30 19:38:02.834275 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834270 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834283 5894 cpu_manager.go:414] "containerMap: removing container" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834290 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834300 5894 cpu_manager.go:414] "containerMap: removing container" podUID="8db11496-7f7c-49f1-ba40-08248a5b2db2" containerName="istio-proxy" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834309 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="8db11496-7f7c-49f1-ba40-08248a5b2db2" containerName="istio-proxy" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834319 5894 cpu_manager.go:414] "containerMap: removing container" podUID="9faa41c5-f9c8-4355-a8ce-d5c44409e061" containerName="maas-api" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834327 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9faa41c5-f9c8-4355-a8ce-d5c44409e061" containerName="maas-api" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834345 5894 cpu_manager.go:414] "containerMap: removing container" podUID="bffc276a-bd01-4f6f-a9f2-252955fa734a" containerName="maas-api" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834352 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="bffc276a-bd01-4f6f-a9f2-252955fa734a" containerName="maas-api" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834363 5894 cpu_manager.go:414] "containerMap: removing container" podUID="49d04e0a-032d-4491-a4ea-be136c886f8a" containerName="maas-api" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834370 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="49d04e0a-032d-4491-a4ea-be136c886f8a" containerName="maas-api" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834388 5894 cpu_manager.go:414] "containerMap: removing container" podUID="eb9bc7c3-4527-4569-b896-b302d1bc5afc" containerName="payload-pre-processing" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834397 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="eb9bc7c3-4527-4569-b896-b302d1bc5afc" containerName="payload-pre-processing" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834411 5894 cpu_manager.go:414] "containerMap: removing container" podUID="d7439fb6-eee6-483a-b43b-1f5bef9b1d49" containerName="maas-api" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834418 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="d7439fb6-eee6-483a-b43b-1f5bef9b1d49" containerName="maas-api" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834435 5894 cpu_manager.go:414] "containerMap: removing container" podUID="b0479234-1db6-465f-9454-54d777b09538" containerName="istio-proxy" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834442 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0479234-1db6-465f-9454-54d777b09538" containerName="istio-proxy" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834464 5894 cpu_manager.go:414] "containerMap: removing container" podUID="f247ccba-3633-4301-b8f2-41a0a8eab7ea" containerName="istio-proxy" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834471 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f247ccba-3633-4301-b8f2-41a0a8eab7ea" containerName="istio-proxy" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834482 5894 cpu_manager.go:414] "containerMap: removing container" podUID="5264ce31-dc5b-42ed-a76e-631aee315933" containerName="manager" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834489 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5264ce31-dc5b-42ed-a76e-631aee315933" containerName="manager" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834501 5894 cpu_manager.go:414] "containerMap: removing container" podUID="1b395010-277d-42b3-b191-0e24f69156c1" containerName="payload-processing" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834508 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1b395010-277d-42b3-b191-0e24f69156c1" containerName="payload-processing" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834519 5894 cpu_manager.go:414] "containerMap: removing container" podUID="9e867658-2d43-4edc-9d26-f23fdd765a80" containerName="istio-proxy" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834527 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9e867658-2d43-4edc-9d26-f23fdd765a80" containerName="istio-proxy" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834544 5894 cpu_manager.go:414] "containerMap: removing container" podUID="b0479234-1db6-465f-9454-54d777b09538" containerName="istio-proxy" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834551 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="b0479234-1db6-465f-9454-54d777b09538" containerName="istio-proxy" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834562 5894 cpu_manager.go:414] "containerMap: removing container" podUID="a37b4ddc-1a56-4ebe-a548-d2681502698a" containerName="istio-proxy" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834569 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="a37b4ddc-1a56-4ebe-a548-d2681502698a" containerName="istio-proxy" Oct 30 19:38:02.834691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834701 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="e1e51799-7dbd-47ed-9b66-e4e355cc03b7" containerName="main" Oct 30 19:38:02.836133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834714 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="8db11496-7f7c-49f1-ba40-08248a5b2db2" containerName="istio-proxy" Oct 30 19:38:02.836133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834723 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="9faa41c5-f9c8-4355-a8ce-d5c44409e061" containerName="maas-api" Oct 30 19:38:02.836133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834736 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="bffc276a-bd01-4f6f-a9f2-252955fa734a" containerName="maas-api" Oct 30 19:38:02.836133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834745 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="49d04e0a-032d-4491-a4ea-be136c886f8a" containerName="maas-api" Oct 30 19:38:02.836133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834755 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="eb9bc7c3-4527-4569-b896-b302d1bc5afc" containerName="payload-pre-processing" Oct 30 19:38:02.836133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834765 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="d7439fb6-eee6-483a-b43b-1f5bef9b1d49" containerName="maas-api" Oct 30 19:38:02.836133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834774 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="b0479234-1db6-465f-9454-54d777b09538" containerName="istio-proxy" Oct 30 19:38:02.836133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834784 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="f247ccba-3633-4301-b8f2-41a0a8eab7ea" containerName="istio-proxy" Oct 30 19:38:02.836133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834795 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="5264ce31-dc5b-42ed-a76e-631aee315933" containerName="manager" Oct 30 19:38:02.836133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834804 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="1b395010-277d-42b3-b191-0e24f69156c1" containerName="payload-processing" Oct 30 19:38:02.836133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834814 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="9e867658-2d43-4edc-9d26-f23fdd765a80" containerName="istio-proxy" Oct 30 19:38:02.836133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834824 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="b0479234-1db6-465f-9454-54d777b09538" containerName="istio-proxy" Oct 30 19:38:02.836133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834831 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="a37b4ddc-1a56-4ebe-a548-d2681502698a" containerName="istio-proxy" Oct 30 19:38:02.836133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834840 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="3c2df19e-78b6-4db7-bc32-db2bef552a2a" containerName="main" Oct 30 19:38:02.836133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834851 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="5935a99f-0b26-4651-a937-d76762e3ce55" containerName="main" Oct 30 19:38:02.836133 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.834858 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="4c72bcc9-7401-4b86-900d-558b768d8d16" containerName="main" Oct 30 19:38:02.838755 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.838735 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq" Oct 30 19:38:02.869363 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.869334 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vp476\" (UniqueName: \"kubernetes.io/projected/185ae304-6486-4e90-856f-a7624a60cd64-kube-api-access-vp476\") pod \"maas-api-revoke-sub-f03629db-s7hdq\" (UID: \"185ae304-6486-4e90-856f-a7624a60cd64\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq" Oct 30 19:38:02.869476 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.869374 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/185ae304-6486-4e90-856f-a7624a60cd64-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-s7hdq\" (UID: \"185ae304-6486-4e90-856f-a7624a60cd64\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq" Oct 30 19:38:02.869476 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.869393 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/185ae304-6486-4e90-856f-a7624a60cd64-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-s7hdq\" (UID: \"185ae304-6486-4e90-856f-a7624a60cd64\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq" Oct 30 19:38:02.891528 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.891496 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 19:38:02.941380 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.941345 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nqq9m\"" Oct 30 19:38:02.942674 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.942651 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq"] Oct 30 19:38:02.970007 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.969973 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/185ae304-6486-4e90-856f-a7624a60cd64-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-s7hdq\" (UID: \"185ae304-6486-4e90-856f-a7624a60cd64\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq" Oct 30 19:38:02.970150 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.970013 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/185ae304-6486-4e90-856f-a7624a60cd64-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-s7hdq\" (UID: \"185ae304-6486-4e90-856f-a7624a60cd64\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq" Oct 30 19:38:02.970150 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.970095 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vp476\" (UniqueName: \"kubernetes.io/projected/185ae304-6486-4e90-856f-a7624a60cd64-kube-api-access-vp476\") pod \"maas-api-revoke-sub-f03629db-s7hdq\" (UID: \"185ae304-6486-4e90-856f-a7624a60cd64\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq" Oct 30 19:38:02.970817 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.970791 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/185ae304-6486-4e90-856f-a7624a60cd64-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-f03629db-s7hdq\" (UID: \"185ae304-6486-4e90-856f-a7624a60cd64\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq" Oct 30 19:38:02.972344 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:02.972322 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/185ae304-6486-4e90-856f-a7624a60cd64-maas-api-xks-ca\") pod \"maas-api-revoke-sub-f03629db-s7hdq\" (UID: \"185ae304-6486-4e90-856f-a7624a60cd64\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq" Oct 30 19:38:03.026981 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:03.026953 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vp476\" (UniqueName: \"kubernetes.io/projected/185ae304-6486-4e90-856f-a7624a60cd64-kube-api-access-vp476\") pod \"maas-api-revoke-sub-f03629db-s7hdq\" (UID: \"185ae304-6486-4e90-856f-a7624a60cd64\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq" Oct 30 19:38:03.150622 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:03.150557 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq" Oct 30 19:38:03.532042 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:03.531943 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq"] Oct 30 19:38:03.534847 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:38:03.534819 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod185ae304_6486_4e90_856f_a7624a60cd64.slice/crio-1e1d2d01e7f76a5c9bf6fe11c6feddd1cf662230fdc8de43ca6cfe242d044d58 WatchSource:0}: Error finding container 1e1d2d01e7f76a5c9bf6fe11c6feddd1cf662230fdc8de43ca6cfe242d044d58: Status 404 returned error can't find the container with id 1e1d2d01e7f76a5c9bf6fe11c6feddd1cf662230fdc8de43ca6cfe242d044d58 Oct 30 19:38:04.317118 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:04.317081 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq" event={"ID":"185ae304-6486-4e90-856f-a7624a60cd64","Type":"ContainerStarted","Data":"9d2b5865dbb2d7ccfa72237484bdd3f201b7382bca52e767e7735f6e32870279"} Oct 30 19:38:04.317482 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:04.317131 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq" event={"ID":"185ae304-6486-4e90-856f-a7624a60cd64","Type":"ContainerStarted","Data":"1e1d2d01e7f76a5c9bf6fe11c6feddd1cf662230fdc8de43ca6cfe242d044d58"} Oct 30 19:38:04.435368 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:04.435321 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq" podStartSLOduration=2.435307136 podStartE2EDuration="2.435307136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:04.434065168 +0000 UTC m=+3554.924192987" watchObservedRunningTime="2026-10-30 19:38:04.435307136 +0000 UTC m=+3554.925434950" Oct 30 19:38:05.323449 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:05.323414 5894 generic.go:356] "Generic (PLEG): container finished" podID="185ae304-6486-4e90-856f-a7624a60cd64" containerID="9d2b5865dbb2d7ccfa72237484bdd3f201b7382bca52e767e7735f6e32870279" exitCode=0 Oct 30 19:38:05.323897 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:05.323476 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq" event={"ID":"185ae304-6486-4e90-856f-a7624a60cd64","Type":"ContainerDied","Data":"9d2b5865dbb2d7ccfa72237484bdd3f201b7382bca52e767e7735f6e32870279"} Oct 30 19:38:06.557734 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:06.557703 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq" Oct 30 19:38:06.603010 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:06.602943 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/185ae304-6486-4e90-856f-a7624a60cd64-kube-api-access-vp476\" (UniqueName: \"kubernetes.io/projected/185ae304-6486-4e90-856f-a7624a60cd64-kube-api-access-vp476\") pod \"185ae304-6486-4e90-856f-a7624a60cd64\" (UID: \"185ae304-6486-4e90-856f-a7624a60cd64\") " Oct 30 19:38:06.603010 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:06.602981 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/185ae304-6486-4e90-856f-a7624a60cd64-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/185ae304-6486-4e90-856f-a7624a60cd64-maas-api-openshift-ca\") pod \"185ae304-6486-4e90-856f-a7624a60cd64\" (UID: \"185ae304-6486-4e90-856f-a7624a60cd64\") " Oct 30 19:38:06.603215 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:06.603013 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/185ae304-6486-4e90-856f-a7624a60cd64-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/185ae304-6486-4e90-856f-a7624a60cd64-maas-api-xks-ca\") pod \"185ae304-6486-4e90-856f-a7624a60cd64\" (UID: \"185ae304-6486-4e90-856f-a7624a60cd64\") " Oct 30 19:38:06.603335 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:06.603312 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/185ae304-6486-4e90-856f-a7624a60cd64-maas-api-openshift-ca" pod "185ae304-6486-4e90-856f-a7624a60cd64" (UID: "185ae304-6486-4e90-856f-a7624a60cd64"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:38:06.605127 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:06.605105 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/185ae304-6486-4e90-856f-a7624a60cd64-kube-api-access-vp476" pod "185ae304-6486-4e90-856f-a7624a60cd64" (UID: "185ae304-6486-4e90-856f-a7624a60cd64"). InnerVolumeSpecName "kube-api-access-vp476". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:38:06.605209 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:06.605138 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/185ae304-6486-4e90-856f-a7624a60cd64-maas-api-xks-ca" pod "185ae304-6486-4e90-856f-a7624a60cd64" (UID: "185ae304-6486-4e90-856f-a7624a60cd64"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:38:06.704545 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:06.704509 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/185ae304-6486-4e90-856f-a7624a60cd64-maas-api-xks-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:38:06.704545 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:06.704539 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vp476\" (UniqueName: \"kubernetes.io/projected/185ae304-6486-4e90-856f-a7624a60cd64-kube-api-access-vp476\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:38:06.704545 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:06.704550 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/185ae304-6486-4e90-856f-a7624a60cd64-maas-api-openshift-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:38:07.332498 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:07.332463 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq" event={"ID":"185ae304-6486-4e90-856f-a7624a60cd64","Type":"ContainerDied","Data":"1e1d2d01e7f76a5c9bf6fe11c6feddd1cf662230fdc8de43ca6cfe242d044d58"} Oct 30 19:38:07.332691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:07.332516 5894 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e1d2d01e7f76a5c9bf6fe11c6feddd1cf662230fdc8de43ca6cfe242d044d58" Oct 30 19:38:07.332691 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:07.332480 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-f03629db-s7hdq" Oct 30 19:38:20.115972 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:38:20.115934 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:38:23.463825 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:23.463788 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx"] Oct 30 19:38:23.464404 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:23.464342 5894 cpu_manager.go:414] "containerMap: removing container" podUID="185ae304-6486-4e90-856f-a7624a60cd64" containerName="revoke-keys" Oct 30 19:38:23.464404 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:23.464362 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="185ae304-6486-4e90-856f-a7624a60cd64" containerName="revoke-keys" Oct 30 19:38:23.464532 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:23.464506 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="8db11496-7f7c-49f1-ba40-08248a5b2db2" containerName="istio-proxy" Oct 30 19:38:23.464532 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:23.464520 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="185ae304-6486-4e90-856f-a7624a60cd64" containerName="revoke-keys" Oct 30 19:38:23.467645 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:23.467624 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx" Oct 30 19:38:23.524818 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:23.524793 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx"] Oct 30 19:38:23.572451 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:23.572419 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 19:38:23.572614 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:23.572509 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nqq9m\"" Oct 30 19:38:23.587695 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:23.587665 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc778\" (UniqueName: \"kubernetes.io/projected/f66e7268-8263-4648-ac07-b22577df2f9b-kube-api-access-rc778\") pod \"maas-api-revoke-sub-0165d7d2-77tfx\" (UID: \"f66e7268-8263-4648-ac07-b22577df2f9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx" Oct 30 19:38:23.587803 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:23.587699 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f66e7268-8263-4648-ac07-b22577df2f9b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-77tfx\" (UID: \"f66e7268-8263-4648-ac07-b22577df2f9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx" Oct 30 19:38:23.587803 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:23.587718 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f66e7268-8263-4648-ac07-b22577df2f9b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-77tfx\" (UID: \"f66e7268-8263-4648-ac07-b22577df2f9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx" Oct 30 19:38:23.688900 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:23.688871 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc778\" (UniqueName: \"kubernetes.io/projected/f66e7268-8263-4648-ac07-b22577df2f9b-kube-api-access-rc778\") pod \"maas-api-revoke-sub-0165d7d2-77tfx\" (UID: \"f66e7268-8263-4648-ac07-b22577df2f9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx" Oct 30 19:38:23.689044 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:23.688907 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f66e7268-8263-4648-ac07-b22577df2f9b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-77tfx\" (UID: \"f66e7268-8263-4648-ac07-b22577df2f9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx" Oct 30 19:38:23.689044 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:23.688927 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f66e7268-8263-4648-ac07-b22577df2f9b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-77tfx\" (UID: \"f66e7268-8263-4648-ac07-b22577df2f9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx" Oct 30 19:38:23.689560 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:23.689528 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f66e7268-8263-4648-ac07-b22577df2f9b-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-0165d7d2-77tfx\" (UID: \"f66e7268-8263-4648-ac07-b22577df2f9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx" Oct 30 19:38:23.691260 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:23.691204 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f66e7268-8263-4648-ac07-b22577df2f9b-maas-api-xks-ca\") pod \"maas-api-revoke-sub-0165d7d2-77tfx\" (UID: \"f66e7268-8263-4648-ac07-b22577df2f9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx" Oct 30 19:38:23.745768 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:23.745703 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc778\" (UniqueName: \"kubernetes.io/projected/f66e7268-8263-4648-ac07-b22577df2f9b-kube-api-access-rc778\") pod \"maas-api-revoke-sub-0165d7d2-77tfx\" (UID: \"f66e7268-8263-4648-ac07-b22577df2f9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx" Oct 30 19:38:23.780663 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:23.780636 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx" Oct 30 19:38:24.151183 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:24.151107 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx"] Oct 30 19:38:24.153536 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:38:24.153510 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf66e7268_8263_4648_ac07_b22577df2f9b.slice/crio-ff3e73181b40252c1cd399f797d67a62f6519763606403803876cccc2c42fd82 WatchSource:0}: Error finding container ff3e73181b40252c1cd399f797d67a62f6519763606403803876cccc2c42fd82: Status 404 returned error can't find the container with id ff3e73181b40252c1cd399f797d67a62f6519763606403803876cccc2c42fd82 Oct 30 19:38:24.420907 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:24.420831 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx" event={"ID":"f66e7268-8263-4648-ac07-b22577df2f9b","Type":"ContainerStarted","Data":"5026186f856bdde81f4ba6d0ede11d8d7a65e9fbc3ccbb86e2c8cd20609fb170"} Oct 30 19:38:24.420907 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:24.420888 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx" event={"ID":"f66e7268-8263-4648-ac07-b22577df2f9b","Type":"ContainerStarted","Data":"ff3e73181b40252c1cd399f797d67a62f6519763606403803876cccc2c42fd82"} Oct 30 19:38:24.537938 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:24.537885 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx" podStartSLOduration=1.537869786 podStartE2EDuration="1.537869786s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-10-30 19:38:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:38:24.536773277 +0000 UTC m=+3575.026901130" watchObservedRunningTime="2026-10-30 19:38:24.537869786 +0000 UTC m=+3575.027997604" Oct 30 19:38:25.426839 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:25.426801 5894 generic.go:356] "Generic (PLEG): container finished" podID="f66e7268-8263-4648-ac07-b22577df2f9b" containerID="5026186f856bdde81f4ba6d0ede11d8d7a65e9fbc3ccbb86e2c8cd20609fb170" exitCode=0 Oct 30 19:38:25.427041 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:25.426868 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx" event={"ID":"f66e7268-8263-4648-ac07-b22577df2f9b","Type":"ContainerDied","Data":"5026186f856bdde81f4ba6d0ede11d8d7a65e9fbc3ccbb86e2c8cd20609fb170"} Oct 30 19:38:26.561820 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:26.561793 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx" Oct 30 19:38:26.715028 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:26.714957 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f66e7268-8263-4648-ac07-b22577df2f9b-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f66e7268-8263-4648-ac07-b22577df2f9b-maas-api-openshift-ca\") pod \"f66e7268-8263-4648-ac07-b22577df2f9b\" (UID: \"f66e7268-8263-4648-ac07-b22577df2f9b\") " Oct 30 19:38:26.715196 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:26.715031 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f66e7268-8263-4648-ac07-b22577df2f9b-kube-api-access-rc778\" (UniqueName: \"kubernetes.io/projected/f66e7268-8263-4648-ac07-b22577df2f9b-kube-api-access-rc778\") pod \"f66e7268-8263-4648-ac07-b22577df2f9b\" (UID: \"f66e7268-8263-4648-ac07-b22577df2f9b\") " Oct 30 19:38:26.715196 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:26.715118 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/f66e7268-8263-4648-ac07-b22577df2f9b-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f66e7268-8263-4648-ac07-b22577df2f9b-maas-api-xks-ca\") pod \"f66e7268-8263-4648-ac07-b22577df2f9b\" (UID: \"f66e7268-8263-4648-ac07-b22577df2f9b\") " Oct 30 19:38:26.715327 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:26.715288 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f66e7268-8263-4648-ac07-b22577df2f9b-maas-api-openshift-ca" pod "f66e7268-8263-4648-ac07-b22577df2f9b" (UID: "f66e7268-8263-4648-ac07-b22577df2f9b"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:38:26.717188 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:26.717159 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f66e7268-8263-4648-ac07-b22577df2f9b-kube-api-access-rc778" pod "f66e7268-8263-4648-ac07-b22577df2f9b" (UID: "f66e7268-8263-4648-ac07-b22577df2f9b"). InnerVolumeSpecName "kube-api-access-rc778". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:38:26.717310 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:26.717220 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f66e7268-8263-4648-ac07-b22577df2f9b-maas-api-xks-ca" pod "f66e7268-8263-4648-ac07-b22577df2f9b" (UID: "f66e7268-8263-4648-ac07-b22577df2f9b"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:38:26.816395 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:26.816358 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/f66e7268-8263-4648-ac07-b22577df2f9b-maas-api-xks-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:38:26.816395 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:26.816388 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/f66e7268-8263-4648-ac07-b22577df2f9b-maas-api-openshift-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:38:26.816395 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:26.816398 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rc778\" (UniqueName: \"kubernetes.io/projected/f66e7268-8263-4648-ac07-b22577df2f9b-kube-api-access-rc778\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:38:27.438391 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:27.438364 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx" Oct 30 19:38:27.438590 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:27.438351 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-0165d7d2-77tfx" event={"ID":"f66e7268-8263-4648-ac07-b22577df2f9b","Type":"ContainerDied","Data":"ff3e73181b40252c1cd399f797d67a62f6519763606403803876cccc2c42fd82"} Oct 30 19:38:27.438590 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:27.438515 5894 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ff3e73181b40252c1cd399f797d67a62f6519763606403803876cccc2c42fd82" Oct 30 19:38:32.214803 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:32.214768 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:38:34.821767 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:34.821730 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2f5rn"] Oct 30 19:38:34.822282 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:34.822137 5894 cpu_manager.go:414] "containerMap: removing container" podUID="f66e7268-8263-4648-ac07-b22577df2f9b" containerName="revoke-keys" Oct 30 19:38:34.822282 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:34.822149 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f66e7268-8263-4648-ac07-b22577df2f9b" containerName="revoke-keys" Oct 30 19:38:34.822282 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:34.822235 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="f66e7268-8263-4648-ac07-b22577df2f9b" containerName="revoke-keys" Oct 30 19:38:34.825411 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:34.825389 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2f5rn" Oct 30 19:38:34.886226 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:34.886184 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/095f2faf-7a75-4e65-ad54-64f40915bcf7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-2f5rn\" (UID: \"095f2faf-7a75-4e65-ad54-64f40915bcf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2f5rn" Oct 30 19:38:34.886408 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:34.886240 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/095f2faf-7a75-4e65-ad54-64f40915bcf7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-2f5rn\" (UID: \"095f2faf-7a75-4e65-ad54-64f40915bcf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2f5rn" Oct 30 19:38:34.886408 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:34.886325 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56zz9\" (UniqueName: \"kubernetes.io/projected/095f2faf-7a75-4e65-ad54-64f40915bcf7-kube-api-access-56zz9\") pod \"maas-api-revoke-sub-6b19338f-2f5rn\" (UID: \"095f2faf-7a75-4e65-ad54-64f40915bcf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2f5rn" Oct 30 19:38:34.928044 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:34.928011 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"opendatahub-ca\"" Oct 30 19:38:34.928203 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:34.928163 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-nqq9m\"" Oct 30 19:38:34.929684 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:34.929665 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2f5rn"] Oct 30 19:38:34.987423 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:34.987393 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/095f2faf-7a75-4e65-ad54-64f40915bcf7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-2f5rn\" (UID: \"095f2faf-7a75-4e65-ad54-64f40915bcf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2f5rn" Oct 30 19:38:34.987423 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:34.987426 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/095f2faf-7a75-4e65-ad54-64f40915bcf7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-2f5rn\" (UID: \"095f2faf-7a75-4e65-ad54-64f40915bcf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2f5rn" Oct 30 19:38:34.987695 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:34.987483 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-56zz9\" (UniqueName: \"kubernetes.io/projected/095f2faf-7a75-4e65-ad54-64f40915bcf7-kube-api-access-56zz9\") pod \"maas-api-revoke-sub-6b19338f-2f5rn\" (UID: \"095f2faf-7a75-4e65-ad54-64f40915bcf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2f5rn" Oct 30 19:38:34.988098 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:34.988077 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/095f2faf-7a75-4e65-ad54-64f40915bcf7-maas-api-openshift-ca\") pod \"maas-api-revoke-sub-6b19338f-2f5rn\" (UID: \"095f2faf-7a75-4e65-ad54-64f40915bcf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2f5rn" Oct 30 19:38:34.989879 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:34.989857 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/095f2faf-7a75-4e65-ad54-64f40915bcf7-maas-api-xks-ca\") pod \"maas-api-revoke-sub-6b19338f-2f5rn\" (UID: \"095f2faf-7a75-4e65-ad54-64f40915bcf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2f5rn" Oct 30 19:38:35.044340 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:35.044307 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56zz9\" (UniqueName: \"kubernetes.io/projected/095f2faf-7a75-4e65-ad54-64f40915bcf7-kube-api-access-56zz9\") pod \"maas-api-revoke-sub-6b19338f-2f5rn\" (UID: \"095f2faf-7a75-4e65-ad54-64f40915bcf7\") " pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2f5rn" Oct 30 19:38:35.138531 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:35.138458 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2f5rn" Oct 30 19:38:35.507925 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:35.507901 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2f5rn"] Oct 30 19:38:35.510710 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:38:35.510673 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod095f2faf_7a75_4e65_ad54_64f40915bcf7.slice/crio-22e77986e784ff21546b6c14888c319f75d944cee75482cd0c631a9c56f8d9fd WatchSource:0}: Error finding container 22e77986e784ff21546b6c14888c319f75d944cee75482cd0c631a9c56f8d9fd: Status 404 returned error can't find the container with id 22e77986e784ff21546b6c14888c319f75d944cee75482cd0c631a9c56f8d9fd Oct 30 19:38:36.479961 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:36.479933 5894 generic.go:356] "Generic (PLEG): container finished" podID="095f2faf-7a75-4e65-ad54-64f40915bcf7" containerID="45c648ac57cbba0a1c384d4c955d1bf81dcbb7e34c3ae17e988988ebc41ceb5c" exitCode=0 Oct 30 19:38:36.480352 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:36.479994 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2f5rn" event={"ID":"095f2faf-7a75-4e65-ad54-64f40915bcf7","Type":"ContainerDied","Data":"45c648ac57cbba0a1c384d4c955d1bf81dcbb7e34c3ae17e988988ebc41ceb5c"} Oct 30 19:38:36.480352 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:36.480037 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2f5rn" event={"ID":"095f2faf-7a75-4e65-ad54-64f40915bcf7","Type":"ContainerStarted","Data":"22e77986e784ff21546b6c14888c319f75d944cee75482cd0c631a9c56f8d9fd"} Oct 30 19:38:37.626935 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:37.626910 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2f5rn" Oct 30 19:38:37.709596 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:37.709548 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/095f2faf-7a75-4e65-ad54-64f40915bcf7-maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/095f2faf-7a75-4e65-ad54-64f40915bcf7-maas-api-xks-ca\") pod \"095f2faf-7a75-4e65-ad54-64f40915bcf7\" (UID: \"095f2faf-7a75-4e65-ad54-64f40915bcf7\") " Oct 30 19:38:37.709747 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:37.709666 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/095f2faf-7a75-4e65-ad54-64f40915bcf7-kube-api-access-56zz9\" (UniqueName: \"kubernetes.io/projected/095f2faf-7a75-4e65-ad54-64f40915bcf7-kube-api-access-56zz9\") pod \"095f2faf-7a75-4e65-ad54-64f40915bcf7\" (UID: \"095f2faf-7a75-4e65-ad54-64f40915bcf7\") " Oct 30 19:38:37.709747 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:37.709693 5894 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/095f2faf-7a75-4e65-ad54-64f40915bcf7-maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/095f2faf-7a75-4e65-ad54-64f40915bcf7-maas-api-openshift-ca\") pod \"095f2faf-7a75-4e65-ad54-64f40915bcf7\" (UID: \"095f2faf-7a75-4e65-ad54-64f40915bcf7\") " Oct 30 19:38:37.710113 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:37.710082 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/095f2faf-7a75-4e65-ad54-64f40915bcf7-maas-api-openshift-ca" pod "095f2faf-7a75-4e65-ad54-64f40915bcf7" (UID: "095f2faf-7a75-4e65-ad54-64f40915bcf7"). InnerVolumeSpecName "maas-api-openshift-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Oct 30 19:38:37.711689 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:37.711662 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/095f2faf-7a75-4e65-ad54-64f40915bcf7-maas-api-xks-ca" pod "095f2faf-7a75-4e65-ad54-64f40915bcf7" (UID: "095f2faf-7a75-4e65-ad54-64f40915bcf7"). InnerVolumeSpecName "maas-api-xks-ca". PluginName "kubernetes.io/secret", VolumeGIDValue "" Oct 30 19:38:37.711816 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:37.711762 5894 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/095f2faf-7a75-4e65-ad54-64f40915bcf7-kube-api-access-56zz9" pod "095f2faf-7a75-4e65-ad54-64f40915bcf7" (UID: "095f2faf-7a75-4e65-ad54-64f40915bcf7"). InnerVolumeSpecName "kube-api-access-56zz9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Oct 30 19:38:37.810801 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:37.810725 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-openshift-ca\" (UniqueName: \"kubernetes.io/configmap/095f2faf-7a75-4e65-ad54-64f40915bcf7-maas-api-openshift-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:38:37.810801 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:37.810749 5894 reconciler_common.go:299] "Volume detached for volume \"maas-api-xks-ca\" (UniqueName: \"kubernetes.io/secret/095f2faf-7a75-4e65-ad54-64f40915bcf7-maas-api-xks-ca\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:38:37.810801 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:37.810758 5894 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56zz9\" (UniqueName: \"kubernetes.io/projected/095f2faf-7a75-4e65-ad54-64f40915bcf7-kube-api-access-56zz9\") on node \"ip-10-0-132-122.us-west-1.compute.internal\" DevicePath \"\"" Oct 30 19:38:38.490316 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:38.490278 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2f5rn" event={"ID":"095f2faf-7a75-4e65-ad54-64f40915bcf7","Type":"ContainerDied","Data":"22e77986e784ff21546b6c14888c319f75d944cee75482cd0c631a9c56f8d9fd"} Oct 30 19:38:38.490316 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:38.490303 5894 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-sub-6b19338f-2f5rn" Oct 30 19:38:38.490544 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:38.490335 5894 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="22e77986e784ff21546b6c14888c319f75d944cee75482cd0c631a9c56f8d9fd" Oct 30 19:38:50.116445 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:38:50.116402 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:38:50.392246 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:50.392222 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 19:38:50.420786 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:50.420760 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7cd56b8dbc-gg799_67b57c01-e4a3-4ae1-ab6e-9b386e181adb/console-operator/1.log" Oct 30 19:38:58.655276 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:58.655245 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-zf2dd_ef60d381-c326-449b-a5f5-47b539535199/manager/0.log" Oct 30 19:38:59.164480 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:59.164450 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-7qw75_43305b9a-736e-472f-ae60-9001cfc5f237/manager/0.log" Oct 30 19:38:59.820479 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:38:59.820444 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-b2qh7_310c3e75-e27c-4094-b0f6-97f99741a1d6/server/0.log" Oct 30 19:39:00.578185 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:00.578158 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb69c88fb-9fph8_2b396120-b7c7-4164-b320-4e4427016800/copy-manifests/0.log" Oct 30 19:39:00.644957 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:00.644924 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb69c88fb-9fph8_2b396120-b7c7-4164-b320-4e4427016800/manager/0.log" Oct 30 19:39:01.552864 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:01.552783 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-7d94f55f4b-9n2r5_3b6fb6f1-df56-4615-b889-13324b291952/proxy/0.log" Oct 30 19:39:02.886018 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:02.885992 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0165d7d2-77tfx_f66e7268-8263-4648-ac07-b22577df2f9b/revoke-keys/0.log" Oct 30 19:39:03.795230 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:03.795203 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-6b19338f-2f5rn_095f2faf-7a75-4e65-ad54-64f40915bcf7/revoke-keys/0.log" Oct 30 19:39:04.230119 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:04.230088 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f03629db-s7hdq_185ae304-6486-4e90-856f-a7624a60cd64/revoke-keys/0.log" Oct 30 19:39:04.457245 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:04.457218 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-nmj5m_4ecfa4cd-fb2a-43b0-9376-8280e2b3e154/postgres/0.log" Oct 30 19:39:06.803296 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:06.803268 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-twqpm_c9b5f0b2-375b-423f-9a97-6c797ca45ab7/manager/0.log" Oct 30 19:39:07.883678 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:07.883631 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-856dd584c5-8hzrm_c4f74804-9c53-4f89-b0e7-50599bb10a9f/manager/0.log" Oct 30 19:39:09.043949 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:09.043913 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-n65j8_55214cf0-106f-45de-953c-fca931bc6a4f/discovery/0.log" Oct 30 19:39:09.324292 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:09.324225 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fb96896c4-jfftn_e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914/kube-auth-proxy/0.log" Oct 30 19:39:10.009482 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:10.009449 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6db586dd6c-7z7fl_75f81dfd-86e7-4532-a20c-42423f908903/istio-proxy/0.log" Oct 30 19:39:11.573410 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:11.573379 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6db586dd6c-nvv4s_3fec8f46-c372-49cc-8c0a-9cb885f465be/istio-proxy/0.log" Oct 30 19:39:11.791175 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:11.791135 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6db586dd6c-w22dg_010f73a2-9a20-49a6-8d88-079a27465e78/istio-proxy/0.log" Oct 30 19:39:12.750806 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:12.750773 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-797b6dc769-r2cdl_3180fff2-f289-47e6-a37c-34c5dff50aa2/router/0.log" Oct 30 19:39:20.117762 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:39:20.117716 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:39:25.943404 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:25.943371 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-776cc66d47-zf2dd_ef60d381-c326-449b-a5f5-47b539535199/manager/0.log" Oct 30 19:39:26.455738 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:26.455702 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-7qw75_43305b9a-736e-472f-ae60-9001cfc5f237/manager/0.log" Oct 30 19:39:26.997674 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:26.997642 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-75bd89fdfc-b2qh7_310c3e75-e27c-4094-b0f6-97f99741a1d6/server/0.log" Oct 30 19:39:27.752643 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:27.752616 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb69c88fb-9fph8_2b396120-b7c7-4164-b320-4e4427016800/copy-manifests/0.log" Oct 30 19:39:27.821065 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:27.821036 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb69c88fb-9fph8_2b396120-b7c7-4164-b320-4e4427016800/manager/0.log" Oct 30 19:39:28.749922 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:28.749890 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_usage-logs-tenancy-proxy-7d94f55f4b-9n2r5_3b6fb6f1-df56-4615-b889-13324b291952/proxy/0.log" Oct 30 19:39:29.882228 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:29.882192 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-0165d7d2-77tfx_f66e7268-8263-4648-ac07-b22577df2f9b/revoke-keys/0.log" Oct 30 19:39:30.763303 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:30.763277 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-6b19338f-2f5rn_095f2faf-7a75-4e65-ad54-64f40915bcf7/revoke-keys/0.log" Oct 30 19:39:31.218398 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:31.218371 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-sub-f03629db-s7hdq_185ae304-6486-4e90-856f-a7624a60cd64/revoke-keys/0.log" Oct 30 19:39:31.461336 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:31.461301 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-7cdbf6dc44-nmj5m_4ecfa4cd-fb2a-43b0-9376-8280e2b3e154/postgres/0.log" Oct 30 19:39:32.545854 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:32.545822 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:33.466163 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:33.466139 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-twqpm_c9b5f0b2-375b-423f-9a97-6c797ca45ab7/manager/0.log" Oct 30 19:39:34.466409 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:34.466381 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-856dd584c5-8hzrm_c4f74804-9c53-4f89-b0e7-50599bb10a9f/manager/0.log" Oct 30 19:39:35.410065 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:35.410027 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-7ddd9cfdc8-n65j8_55214cf0-106f-45de-953c-fca931bc6a4f/discovery/0.log" Oct 30 19:39:35.696543 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:35.696456 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-5fb96896c4-jfftn_e1d14c98-aefa-45c8-a6c2-8dcdfc2e3914/kube-auth-proxy/0.log" Oct 30 19:39:36.354150 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:36.354123 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6db586dd6c-7z7fl_75f81dfd-86e7-4532-a20c-42423f908903/istio-proxy/0.log" Oct 30 19:39:37.872802 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:37.872772 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6db586dd6c-nvv4s_3fec8f46-c372-49cc-8c0a-9cb885f465be/istio-proxy/0.log" Oct 30 19:39:38.085078 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:38.085048 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-6db586dd6c-w22dg_010f73a2-9a20-49a6-8d88-079a27465e78/istio-proxy/0.log" Oct 30 19:39:39.035297 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:39.035266 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-797b6dc769-r2cdl_3180fff2-f289-47e6-a37c-34c5dff50aa2/router/0.log" Oct 30 19:39:39.708374 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:39:39.708335 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:39:50.116912 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:39:50.116874 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:40:20.116724 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:40:20.116681 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:40:32.949829 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:40:32.949795 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:50.117718 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:40:50.117673 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:40:52.501995 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:40:52.501963 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:40:53.879387 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:40:53.879355 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024 Oct 30 19:41:02.488105 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:02.488069 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-hmzzs_8c9b04e2-b099-4704-8786-b64a8991d0f4/global-pull-secret-syncer/0.log" Oct 30 19:41:03.713888 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:03.713858 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-mm49z_7a4adea8-c5dc-4a4d-ad54-e985838fa6c5/connection-checker/0.log" Oct 30 19:41:04.131948 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:04.131861 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-pvzsh_a36f2ae5-c6ca-49cf-a775-82a97e92f2c4/connection-checker/0.log" Oct 30 19:41:04.551006 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:04.550976 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kas-connection-checker-685895c7c9-vwws2_34a1d011-b30b-49b8-bc75-bfb9dbba3817/connection-checker/0.log" Oct 30 19:41:04.991449 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:04.991419 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" Oct 30 19:41:06.780718 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:06.780684 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-122.us-west-1.compute.internal_99a39564afa8d640f032873b7cd4dd97/haproxy/0.log" Oct 30 19:41:11.980246 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:11.980216 5894 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf"] Oct 30 19:41:11.980664 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:11.980649 5894 cpu_manager.go:414] "containerMap: removing container" podUID="095f2faf-7a75-4e65-ad54-64f40915bcf7" containerName="revoke-keys" Oct 30 19:41:11.980714 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:11.980666 5894 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="095f2faf-7a75-4e65-ad54-64f40915bcf7" containerName="revoke-keys" Oct 30 19:41:11.980756 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:11.980747 5894 memory_manager.go:365] "RemoveStaleState removing state" podUID="095f2faf-7a75-4e65-ad54-64f40915bcf7" containerName="revoke-keys" Oct 30 19:41:11.983810 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:11.983794 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" Oct 30 19:41:12.006473 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.006451 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-65bfd79598-twqpm_c9b5f0b2-375b-423f-9a97-6c797ca45ab7/manager/0.log" Oct 30 19:41:12.037114 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.037093 5894 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-vh98j\"/\"default-dockercfg-wmjvf\"" Oct 30 19:41:12.086086 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.086055 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vh98j\"/\"kube-root-ca.crt\"" Oct 30 19:41:12.086086 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.086074 5894 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-vh98j\"/\"openshift-service-ca.crt\"" Oct 30 19:41:12.087708 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.087686 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf"] Oct 30 19:41:12.122460 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.122411 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ae0162a8-1977-46c8-8b86-60131dbce5f1-lib-modules\") pod \"perf-node-gather-daemonset-rzdcf\" (UID: \"ae0162a8-1977-46c8-8b86-60131dbce5f1\") " pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" Oct 30 19:41:12.122640 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.122511 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hflf6\" (UniqueName: \"kubernetes.io/projected/ae0162a8-1977-46c8-8b86-60131dbce5f1-kube-api-access-hflf6\") pod \"perf-node-gather-daemonset-rzdcf\" (UID: \"ae0162a8-1977-46c8-8b86-60131dbce5f1\") " pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" Oct 30 19:41:12.122640 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.122548 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ae0162a8-1977-46c8-8b86-60131dbce5f1-podres\") pod \"perf-node-gather-daemonset-rzdcf\" (UID: \"ae0162a8-1977-46c8-8b86-60131dbce5f1\") " pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" Oct 30 19:41:12.122640 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.122605 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae0162a8-1977-46c8-8b86-60131dbce5f1-sys\") pod \"perf-node-gather-daemonset-rzdcf\" (UID: \"ae0162a8-1977-46c8-8b86-60131dbce5f1\") " pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" Oct 30 19:41:12.122930 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.122895 5894 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ae0162a8-1977-46c8-8b86-60131dbce5f1-proc\") pod \"perf-node-gather-daemonset-rzdcf\" (UID: \"ae0162a8-1977-46c8-8b86-60131dbce5f1\") " pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" Oct 30 19:41:12.223535 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.223499 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ae0162a8-1977-46c8-8b86-60131dbce5f1-lib-modules\") pod \"perf-node-gather-daemonset-rzdcf\" (UID: \"ae0162a8-1977-46c8-8b86-60131dbce5f1\") " pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" Oct 30 19:41:12.223739 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.223540 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hflf6\" (UniqueName: \"kubernetes.io/projected/ae0162a8-1977-46c8-8b86-60131dbce5f1-kube-api-access-hflf6\") pod \"perf-node-gather-daemonset-rzdcf\" (UID: \"ae0162a8-1977-46c8-8b86-60131dbce5f1\") " pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" Oct 30 19:41:12.223739 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.223567 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ae0162a8-1977-46c8-8b86-60131dbce5f1-podres\") pod \"perf-node-gather-daemonset-rzdcf\" (UID: \"ae0162a8-1977-46c8-8b86-60131dbce5f1\") " pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" Oct 30 19:41:12.223739 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.223650 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ae0162a8-1977-46c8-8b86-60131dbce5f1-lib-modules\") pod \"perf-node-gather-daemonset-rzdcf\" (UID: \"ae0162a8-1977-46c8-8b86-60131dbce5f1\") " pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" Oct 30 19:41:12.223739 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.223661 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/ae0162a8-1977-46c8-8b86-60131dbce5f1-podres\") pod \"perf-node-gather-daemonset-rzdcf\" (UID: \"ae0162a8-1977-46c8-8b86-60131dbce5f1\") " pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" Oct 30 19:41:12.223739 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.223698 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae0162a8-1977-46c8-8b86-60131dbce5f1-sys\") pod \"perf-node-gather-daemonset-rzdcf\" (UID: \"ae0162a8-1977-46c8-8b86-60131dbce5f1\") " pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" Oct 30 19:41:12.223944 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.223757 5894 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ae0162a8-1977-46c8-8b86-60131dbce5f1-proc\") pod \"perf-node-gather-daemonset-rzdcf\" (UID: \"ae0162a8-1977-46c8-8b86-60131dbce5f1\") " pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" Oct 30 19:41:12.223944 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.223807 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ae0162a8-1977-46c8-8b86-60131dbce5f1-sys\") pod \"perf-node-gather-daemonset-rzdcf\" (UID: \"ae0162a8-1977-46c8-8b86-60131dbce5f1\") " pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" Oct 30 19:41:12.223944 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.223831 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/ae0162a8-1977-46c8-8b86-60131dbce5f1-proc\") pod \"perf-node-gather-daemonset-rzdcf\" (UID: \"ae0162a8-1977-46c8-8b86-60131dbce5f1\") " pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" Oct 30 19:41:12.281318 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.281253 5894 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hflf6\" (UniqueName: \"kubernetes.io/projected/ae0162a8-1977-46c8-8b86-60131dbce5f1-kube-api-access-hflf6\") pod \"perf-node-gather-daemonset-rzdcf\" (UID: \"ae0162a8-1977-46c8-8b86-60131dbce5f1\") " pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" Oct 30 19:41:12.295931 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.295906 5894 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" Oct 30 19:41:12.678127 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:12.678082 5894 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf"] Oct 30 19:41:12.680960 ip-10-0-132-122 kubenswrapper[5894]: W1030 19:41:12.680926 5894 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podae0162a8_1977_46c8_8b86_60131dbce5f1.slice/crio-efc755a079f6225340556fae9debf53ffdb6c6f5a3387103fb9d6986fc60162b WatchSource:0}: Error finding container efc755a079f6225340556fae9debf53ffdb6c6f5a3387103fb9d6986fc60162b: Status 404 returned error can't find the container with id efc755a079f6225340556fae9debf53ffdb6c6f5a3387103fb9d6986fc60162b Oct 30 19:41:13.219658 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:13.219624 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" event={"ID":"ae0162a8-1977-46c8-8b86-60131dbce5f1","Type":"ContainerStarted","Data":"67d4f0d96443daedbe8a42990d3825d96dd1cbd01d50e03a7ff01bba7d0eee76"} Oct 30 19:41:13.220112 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:13.219670 5894 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" event={"ID":"ae0162a8-1977-46c8-8b86-60131dbce5f1","Type":"ContainerStarted","Data":"efc755a079f6225340556fae9debf53ffdb6c6f5a3387103fb9d6986fc60162b"} Oct 30 19:41:13.220112 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:13.219700 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" Oct 30 19:41:13.341229 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:13.341180 5894 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" podStartSLOduration=2.341166966 podStartE2EDuration="2.341166966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-10-30 19:41:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-10-30 19:41:13.340560175 +0000 UTC m=+3743.830687995" watchObservedRunningTime="2026-10-30 19:41:13.341166966 +0000 UTC m=+3743.831294785" Oct 30 19:41:13.737236 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:13.737202 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-856dd584c5-8hzrm_c4f74804-9c53-4f89-b0e7-50599bb10a9f/manager/0.log" Oct 30 19:41:19.234229 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:19.234196 5894 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-vh98j/perf-node-gather-daemonset-rzdcf" Oct 30 19:41:19.342096 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:19.342063 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-884576c57-td46q_2797391b-fd3d-4222-82c8-b28a2d16c9e3/cluster-monitoring-operator/0.log" Oct 30 19:41:20.116645 ip-10-0-132-122 kubenswrapper[5894]: E1030 19:41:20.116609 5894 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-5bn4v" containerName="extractor" Oct 30 19:41:23.116466 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:23.116427 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k4tjq_7b46058e-2df9-4fa1-bd75-7d24f1229256/node-exporter/0.log" Oct 30 19:41:23.530470 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:23.530444 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k4tjq_7b46058e-2df9-4fa1-bd75-7d24f1229256/kube-rbac-proxy/0.log" Oct 30 19:41:23.944699 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:23.944672 5894 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k4tjq_7b46058e-2df9-4fa1-bd75-7d24f1229256/init-textfile/0.log" Oct 30 19:41:33.223784 ip-10-0-132-122 kubenswrapper[5894]: I1030 19:41:33.223753 5894 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bchfv_4b9a31e1-73c6-4617-ac72-296e03e928f2/konnectivity-agent/0.log" limit=1024